[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115480851507-30472-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 15:11:20 dist-test-slave-42z9 krb5kdc[30481](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 15:11:20 dist-test-slave-42z9 krb5kdc[30481](info): set up 2 sockets
Aug 07 15:11:20 dist-test-slave-42z9 krb5kdc[30481](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260807 15:11:22.048462 30472 mini_kdc.cc:121] Time spent starting KDC: real 1.177s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115480851507-30472-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115480851507-30472-0/krb5kdc/test-user.keytab.
Aug 07 15:11:22 dist-test-slave-42z9 krb5kdc[30481](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115482, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T15:11:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T15:11:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.194.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.194.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.194.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.62.keytab.
Entry for principal kudu/127.29.194.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.62.keytab.
I20260807 15:11:22.189719 30472 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7fZSIL/build/release/bin/kudu
/tmp/dist-test-task7fZSIL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115480851507-30472-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115480851507-30472-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115480851507-30472-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115480851507-30472-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.194.62:45901
--webserver_interface=127.29.194.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.62.keytab
--principal=kudu/127.29.194.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.194.20:45795
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.194.62:45901
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115480851507-30472-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115480851507-30472-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115480851507-30472-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:11:22.279525 30500 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:11:22.279774 30500 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:11:22.279800 30500 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:11:22.281023 30500 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:11:22.281078 30500 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:11:22.281091 30500 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:11:22.281103 30500 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:11:22.281114 30500 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:11:22.282735 30500 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.194.20:45795
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115480851507-30472-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115480851507-30472-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.194.62:45901
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.62.keytab
--principal=kudu/127.29.194.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.194.62:45901
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115480851507-30472-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.194.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30500
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115480851507-30472-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:11:22.282961 30500 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:11:22.283234 30500 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:11:22.288450 30507 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:11:22.288450 30513 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:11:22.288584 30500 server_base.cc:1034] running on GCE node
W20260807 15:11:22.288445 30506 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:11:22.288905 30500 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:11:22.289301 30500 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:11:22.290469 30500 hybrid_clock.cc:648] HybridClock initialized: now 1786115482290449 us; error 36 us; skew 500 ppm
Aug 07 15:11:22 dist-test-slave-42z9 krb5kdc[30481](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115482, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.194.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:11:22.293045 30500 init.cc:377] Logged in from keytab as kudu/127.29.194.62@KRBTEST.COM (short username kudu)
I20260807 15:11:22.293515 30500 webserver.cc:460] Webserver started at http://127.29.194.62:46393/ using document root <none> and password file <none>
I20260807 15:11:22.293766 30500 fs_manager.cc:359] Metadata directory not provided
I20260807 15:11:22.293818 30500 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:11:22.293943 30500 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:11:22.295125 30500 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115480851507-30472-0/minicluster-data/master-0/data/instance:
uuid: "27e6c5955cea480a98aa4851ed86a659"
format_stamp: "Formatted at 2026-08-07 15:11:22 on dist-test-slave-42z9"
server_key: "34495ee2feddb8fe088ee54a586b56d8"
server_key_iv: "eee3f3e1758832cdceaff80e693c7a71"
server_key_version: "encryptionkey@0"
I20260807 15:11:22.295463 30500 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115480851507-30472-0/minicluster-data/master-0/wal/instance:
uuid: "27e6c5955cea480a98aa4851ed86a659"
format_stamp: "Formatted at 2026-08-07 15:11:22 on dist-test-slave-42z9"
server_key: "34495ee2feddb8fe088ee54a586b56d8"
server_key_iv: "eee3f3e1758832cdceaff80e693c7a71"
server_key_version: "encryptionkey@0"
I20260807 15:11:22.297025 30500 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260807 15:11:22.297798 30526 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:11:22.298009 30500 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260807 15:11:22.298101 30500 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115480851507-30472-0/minicluster-data/master-0/data,/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115480851507-30472-0/minicluster-data/master-0/wal
uuid: "27e6c5955cea480a98aa4851ed86a659"
format_stamp: "Formatted at 2026-08-07 15:11:22 on dist-test-slave-42z9"
server_key: "34495ee2feddb8fe088ee54a586b56d8"
server_key_iv: "eee3f3e1758832cdceaff80e693c7a71"
server_key_version: "encryptionkey@0"
I20260807 15:11:22.298168 30500 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115480851507-30472-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115480851507-30472-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115480851507-30472-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:11:22.312873 30500 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:11:22.315306 30500 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:11:22.315459 30500 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:11:22.321517 30500 rpc_server.cc:307] RPC server started. Bound to: 127.29.194.62:45901
I20260807 15:11:22.321905 30601 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.194.62:45901 every 8 connection(s)
I20260807 15:11:22.321947 30500 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115480851507-30472-0/minicluster-data/master-0/data/info.pb
I20260807 15:11:22.322698 30602 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:11:22.325706 30602 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 27e6c5955cea480a98aa4851ed86a659: Bootstrap starting.
I20260807 15:11:22.326642 30602 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 27e6c5955cea480a98aa4851ed86a659: Neither blocks nor log segments found. Creating new log.
I20260807 15:11:22.327095 30602 log.cc:824] T 00000000000000000000000000000000 P 27e6c5955cea480a98aa4851ed86a659: Log is configured to *not* fsync() on all Append() calls
I20260807 15:11:22.327071 30472 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7fZSIL/build/release/bin/kudu as pid 30500
I20260807 15:11:22.327205 30472 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115480851507-30472-0/minicluster-data/master-0/wal/instance
I20260807 15:11:22.327998 30472 external_mini_cluster.cc:1428] Setting key 1e6374c8d4f792d422a4cf6072417cf2
I20260807 15:11:22.328280 30602 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 27e6c5955cea480a98aa4851ed86a659: No bootstrap required, opened a new log
I20260807 15:11:22.330878 30602 raft_consensus.cc:348] T 00000000000000000000000000000000 P 27e6c5955cea480a98aa4851ed86a659 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27e6c5955cea480a98aa4851ed86a659" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 45901 } }
I20260807 15:11:22.331115 30602 raft_consensus.cc:374] T 00000000000000000000000000000000 P 27e6c5955cea480a98aa4851ed86a659 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:11:22.331159 30602 raft_consensus.cc:729] T 00000000000000000000000000000000 P 27e6c5955cea480a98aa4851ed86a659 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 27e6c5955cea480a98aa4851ed86a659, State: Initialized, Role: FOLLOWER
I20260807 15:11:22.331305 30602 consensus_queue.cc:260] T 00000000000000000000000000000000 P 27e6c5955cea480a98aa4851ed86a659 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27e6c5955cea480a98aa4851ed86a659" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 45901 } }
I20260807 15:11:22.331398 30602 raft_consensus.cc:388] T 00000000000000000000000000000000 P 27e6c5955cea480a98aa4851ed86a659 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:11:22.331430 30602 raft_consensus.cc:482] T 00000000000000000000000000000000 P 27e6c5955cea480a98aa4851ed86a659 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:11:22.331487 30602 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 27e6c5955cea480a98aa4851ed86a659 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:11:22.333042 30602 raft_consensus.cc:504] T 00000000000000000000000000000000 P 27e6c5955cea480a98aa4851ed86a659 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27e6c5955cea480a98aa4851ed86a659" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 45901 } }
I20260807 15:11:22.333174 30602 leader_election.cc:302] T 00000000000000000000000000000000 P 27e6c5955cea480a98aa4851ed86a659 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 27e6c5955cea480a98aa4851ed86a659; no voters:
I20260807 15:11:22.333356 30602 leader_election.cc:288] T 00000000000000000000000000000000 P 27e6c5955cea480a98aa4851ed86a659 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 15:11:22.333402 30608 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 27e6c5955cea480a98aa4851ed86a659 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:11:22.333628 30608 raft_consensus.cc:686] T 00000000000000000000000000000000 P 27e6c5955cea480a98aa4851ed86a659 [term 1 LEADER]: Becoming Leader. State: Replica: 27e6c5955cea480a98aa4851ed86a659, State: Running, Role: LEADER
I20260807 15:11:22.333732 30602 sys_catalog.cc:564] T 00000000000000000000000000000000 P 27e6c5955cea480a98aa4851ed86a659 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:11:22.333729 30608 consensus_queue.cc:237] T 00000000000000000000000000000000 P 27e6c5955cea480a98aa4851ed86a659 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27e6c5955cea480a98aa4851ed86a659" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 45901 } }
I20260807 15:11:22.334312 30610 sys_catalog.cc:455] T 00000000000000000000000000000000 P 27e6c5955cea480a98aa4851ed86a659 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 27e6c5955cea480a98aa4851ed86a659. Latest consensus state: current_term: 1 leader_uuid: "27e6c5955cea480a98aa4851ed86a659" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27e6c5955cea480a98aa4851ed86a659" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 45901 } } }
I20260807 15:11:22.334419 30610 sys_catalog.cc:458] T 00000000000000000000000000000000 P 27e6c5955cea480a98aa4851ed86a659 [sys.catalog]: This master's current role is: LEADER
I20260807 15:11:22.334609 30609 sys_catalog.cc:455] T 00000000000000000000000000000000 P 27e6c5955cea480a98aa4851ed86a659 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "27e6c5955cea480a98aa4851ed86a659" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27e6c5955cea480a98aa4851ed86a659" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 45901 } } }
I20260807 15:11:22.334693 30609 sys_catalog.cc:458] T 00000000000000000000000000000000 P 27e6c5955cea480a98aa4851ed86a659 [sys.catalog]: This master's current role is: LEADER
I20260807 15:11:22.335158 30621 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:11:22.335705 30621 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:11:22.337491 30621 catalog_manager.cc:1347] Generated new cluster ID: e455ac6f95f9476dba609bc22a7e0556
I20260807 15:11:22.337565 30621 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 07 15:11:22 dist-test-slave-42z9 krb5kdc[30481](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115482, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.194.62@KRBTEST.COM
I20260807 15:11:22.347739 30606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:22.329428 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46500 (local address 127.29.194.62:45901)
0807 15:11:22.329744 (+ 316us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:22.329756 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:22.330108 (+ 352us) server_negotiation.cc:407] Connection header received
0807 15:11:22.332511 (+ 2403us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:22.332517 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:22.332563 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:22.332679 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:11:22.335001 (+ 2322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:22.336388 (+ 1387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:22.337954 (+ 1566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:22.338603 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:22.338730 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:22.343305 (+ 4575us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:11:22.343312 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:11:22.343312 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:11:22.343326 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:11:22.345388 (+ 2062us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:22.345995 (+ 607us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:22.345998 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:22.345998 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:22.346136 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:22.346424 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:22.346425 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:22.346425 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:22.346733 (+ 308us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:11:22.346896 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0807 15:11:22.347209 (+ 313us) server_negotiation.cc:299] Negotiation successful
0807 15:11:22.347356 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":204,"thread_start_us":117,"threads_started":1}
I20260807 15:11:22.353003 30621 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:11:22.353690 30621 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:11:22.359077 30621 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 27e6c5955cea480a98aa4851ed86a659: Generated new TSK 0
I20260807 15:11:22.359289 30621 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.29.194.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.194.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.194.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.1.keytab.
Entry for principal kudu/127.29.194.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.1.keytab.
I20260807 15:11:22.393370 30472 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7fZSIL/build/release/bin/kudu
/tmp/dist-test-task7fZSIL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115480851507-30472-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115480851507-30472-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115480851507-30472-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115480851507-30472-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.194.1:0
--local_ip_for_outbound_sockets=127.29.194.1
--webserver_interface=127.29.194.1
--webserver_port=0
--tserver_master_addrs=127.29.194.62:45901
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.1.keytab
--principal=kudu/127.29.194.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.194.20:45795
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115480851507-30472-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115480851507-30472-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115480851507-30472-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:11:22.488296 30638 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:11:22.488641 30638 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:11:22.488677 30638 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:11:22.490584 30638 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:11:22.490653 30638 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:11:22.490690 30638 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.194.1
I20260807 15:11:22.492650 30638 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.194.20:45795
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115480851507-30472-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115480851507-30472-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.1.keytab
--principal=kudu/127.29.194.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.194.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115480851507-30472-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.194.1
--webserver_port=0
--tserver_master_addrs=127.29.194.62:45901
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30638
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.194.1
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115480851507-30472-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:11:22.492966 30638 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:11:22.493232 30638 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:11:22.499720 30644 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:11:22.499720 30647 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:11:22.499782 30645 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:11:22.500013 30638 server_base.cc:1034] running on GCE node
I20260807 15:11:22.500283 30638 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:11:22.500586 30638 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:11:22.501756 30638 hybrid_clock.cc:648] HybridClock initialized: now 1786115482501735 us; error 54 us; skew 500 ppm
Aug 07 15:11:22 dist-test-slave-42z9 krb5kdc[30481](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115482, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.194.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:11:22.504565 30638 init.cc:377] Logged in from keytab as kudu/127.29.194.1@KRBTEST.COM (short username kudu)
I20260807 15:11:22.505079 30638 webserver.cc:460] Webserver started at http://127.29.194.1:36701/ using document root <none> and password file <none>
I20260807 15:11:22.505316 30638 fs_manager.cc:359] Metadata directory not provided
I20260807 15:11:22.505365 30638 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:11:22.505545 30638 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:11:22.506687 30638 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115480851507-30472-0/minicluster-data/ts-0/data/instance:
uuid: "f286469f57934a099100582a3e1f61f5"
format_stamp: "Formatted at 2026-08-07 15:11:22 on dist-test-slave-42z9"
server_key: "d0acc327898f72809888a4e4081d8949"
server_key_iv: "f9e44f068052648ab77fa4881461107e"
server_key_version: "encryptionkey@0"
I20260807 15:11:22.507046 30638 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115480851507-30472-0/minicluster-data/ts-0/wal/instance:
uuid: "f286469f57934a099100582a3e1f61f5"
format_stamp: "Formatted at 2026-08-07 15:11:22 on dist-test-slave-42z9"
server_key: "d0acc327898f72809888a4e4081d8949"
server_key_iv: "f9e44f068052648ab77fa4881461107e"
server_key_version: "encryptionkey@0"
I20260807 15:11:22.508505 30638 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 15:11:22.509305 30656 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:11:22.509534 30638 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:11:22.509635 30638 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115480851507-30472-0/minicluster-data/ts-0/data,/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115480851507-30472-0/minicluster-data/ts-0/wal
uuid: "f286469f57934a099100582a3e1f61f5"
format_stamp: "Formatted at 2026-08-07 15:11:22 on dist-test-slave-42z9"
server_key: "d0acc327898f72809888a4e4081d8949"
server_key_iv: "f9e44f068052648ab77fa4881461107e"
server_key_version: "encryptionkey@0"
I20260807 15:11:22.509704 30638 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115480851507-30472-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115480851507-30472-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115480851507-30472-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:11:22.534734 30638 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:11:22.537110 30638 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:11:22.537268 30638 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:11:22.537577 30638 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:11:22.537938 30638 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:11:22.537976 30638 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:11:22.538007 30638 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:11:22.538105 30638 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:11:22.546016 30638 rpc_server.cc:307] RPC server started. Bound to: 127.29.194.1:33079
I20260807 15:11:22.546085 30837 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.194.1:33079 every 8 connection(s)
I20260807 15:11:22.546429 30638 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115480851507-30472-0/minicluster-data/ts-0/data/info.pb
I20260807 15:11:22.551245 30472 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7fZSIL/build/release/bin/kudu as pid 30638
I20260807 15:11:22.551388 30472 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115480851507-30472-0/minicluster-data/ts-0/wal/instance
I20260807 15:11:22.551585 30472 external_mini_cluster.cc:1428] Setting key fa86e90da3a558aab2a28ece2237a363
Aug 07 15:11:22 dist-test-slave-42z9 krb5kdc[30481](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115482, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.194.1@KRBTEST.COM for kudu/127.29.194.62@KRBTEST.COM
I20260807 15:11:22.557603 30606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:22.547276 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.194.1:47997 (local address 127.29.194.62:45901)
0807 15:11:22.547350 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:22.547352 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:22.547555 (+ 203us) server_negotiation.cc:407] Connection header received
0807 15:11:22.548091 (+ 536us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:22.548092 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:22.548105 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:22.548158 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:11:22.549314 (+ 1156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:22.549901 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:22.550936 (+ 1035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:22.551144 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:22.551203 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:22.553707 (+ 2504us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:11:22.553709 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:11:22.553709 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:11:22.553717 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:11:22.555275 (+ 1558us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:22.556196 (+ 921us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:22.556199 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:22.556199 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:22.556315 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:22.556697 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:22.556698 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:22.556699 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:22.556863 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:11:22.556939 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0807 15:11:22.557354 (+ 415us) server_negotiation.cc:299] Negotiation successful
0807 15:11:22.557515 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260807 15:11:22.557879 30841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:22.547245 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:45901 (local address 127.29.194.1:47997)
0807 15:11:22.547423 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0807 15:11:22.547463 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0807 15:11:22.547987 (+ 524us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:11:22.548233 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:11:22.548237 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:11:22.548982 (+ 745us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:11:22.549199 (+ 217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:22.549205 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:22.549994 (+ 789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:22.549995 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:22.550833 (+ 838us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:22.550835 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:22.551222 (+ 387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:22.551223 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:22.551352 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:22.551930 (+ 578us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:11:22.551941 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:11:22.553600 (+ 1659us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:11:22.555456 (+ 1856us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:11:22.555476 (+ 20us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:11:22.555479 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:11:22.556025 (+ 546us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:11:22.556407 (+ 382us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:11:22.556408 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:11:22.556409 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:11:22.556593 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:11:22.556935 (+ 342us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:11:22.556943 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:11:22.557166 (+ 223us) client_negotiation.cc:769] Sending connection context
0807 15:11:22.557308 (+ 142us) client_negotiation.cc:240] Negotiation successful
0807 15:11:22.557572 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":86,"threads_started":1}
I20260807 15:11:22.558519 30838 heartbeater.cc:347] Connected to a master server at 127.29.194.62:45901
I20260807 15:11:22.558655 30838 heartbeater.cc:464] Registering TS with master...
I20260807 15:11:22.558885 30838 heartbeater.cc:511] Master 127.29.194.62:45901 requested a full tablet report, sending...
I20260807 15:11:22.559759 30552 ts_manager.cc:194] Registered new tserver with Master: f286469f57934a099100582a3e1f61f5 (127.29.194.1:33079)
I20260807 15:11:22.560966 30552 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.194.1@KRBTEST.COM'} at 127.29.194.1:47997
WARNING: no policy specified for kudu/127.29.194.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.194.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.194.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.2.keytab.
Entry for principal kudu/127.29.194.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.2.keytab.
I20260807 15:11:22.586107 30472 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7fZSIL/build/release/bin/kudu
/tmp/dist-test-task7fZSIL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115480851507-30472-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115480851507-30472-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115480851507-30472-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115480851507-30472-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.194.2:0
--local_ip_for_outbound_sockets=127.29.194.2
--webserver_interface=127.29.194.2
--webserver_port=0
--tserver_master_addrs=127.29.194.62:45901
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.2.keytab
--principal=kudu/127.29.194.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.194.20:45795
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115480851507-30472-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115480851507-30472-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115480851507-30472-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:11:22.672504 30846 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:11:22.672730 30846 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:11:22.672756 30846 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:11:22.674065 30846 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:11:22.674114 30846 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:11:22.674144 30846 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.194.2
I20260807 15:11:22.675760 30846 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.194.20:45795
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115480851507-30472-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115480851507-30472-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.2.keytab
--principal=kudu/127.29.194.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.194.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115480851507-30472-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.194.2
--webserver_port=0
--tserver_master_addrs=127.29.194.62:45901
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30846
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.194.2
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115480851507-30472-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:11:22.675982 30846 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:11:22.676168 30846 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:11:22.681643 30856 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:11:22.681761 30846 server_base.cc:1034] running on GCE node
W20260807 15:11:22.681802 30854 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:11:22.681814 30858 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:11:22.682109 30846 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:11:22.682476 30846 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:11:22.683663 30846 hybrid_clock.cc:648] HybridClock initialized: now 1786115482683626 us; error 56 us; skew 500 ppm
Aug 07 15:11:22 dist-test-slave-42z9 krb5kdc[30481](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115482, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.194.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:11:22.686380 30846 init.cc:377] Logged in from keytab as kudu/127.29.194.2@KRBTEST.COM (short username kudu)
I20260807 15:11:22.686900 30846 webserver.cc:460] Webserver started at http://127.29.194.2:36577/ using document root <none> and password file <none>
I20260807 15:11:22.687093 30846 fs_manager.cc:359] Metadata directory not provided
I20260807 15:11:22.687132 30846 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:11:22.687232 30846 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:11:22.688261 30846 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115480851507-30472-0/minicluster-data/ts-1/data/instance:
uuid: "63cdd5e13a484c4bbcac173ec370e7ae"
format_stamp: "Formatted at 2026-08-07 15:11:22 on dist-test-slave-42z9"
server_key: "e3f33dded6ed55d5eeeb62a032c267d2"
server_key_iv: "a2992cb9fab83d4ab5ee46ebf2c491a0"
server_key_version: "encryptionkey@0"
I20260807 15:11:22.688584 30846 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115480851507-30472-0/minicluster-data/ts-1/wal/instance:
uuid: "63cdd5e13a484c4bbcac173ec370e7ae"
format_stamp: "Formatted at 2026-08-07 15:11:22 on dist-test-slave-42z9"
server_key: "e3f33dded6ed55d5eeeb62a032c267d2"
server_key_iv: "a2992cb9fab83d4ab5ee46ebf2c491a0"
server_key_version: "encryptionkey@0"
I20260807 15:11:22.690001 30846 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 15:11:22.690851 30866 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:11:22.691043 30846 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:11:22.691133 30846 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115480851507-30472-0/minicluster-data/ts-1/data,/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115480851507-30472-0/minicluster-data/ts-1/wal
uuid: "63cdd5e13a484c4bbcac173ec370e7ae"
format_stamp: "Formatted at 2026-08-07 15:11:22 on dist-test-slave-42z9"
server_key: "e3f33dded6ed55d5eeeb62a032c267d2"
server_key_iv: "a2992cb9fab83d4ab5ee46ebf2c491a0"
server_key_version: "encryptionkey@0"
I20260807 15:11:22.691190 30846 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115480851507-30472-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115480851507-30472-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115480851507-30472-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:11:22.709010 30846 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:11:22.711233 30846 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:11:22.711386 30846 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:11:22.711612 30846 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:11:22.711952 30846 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:11:22.711987 30846 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:11:22.712010 30846 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:11:22.712021 30846 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:11:22.718921 30846 rpc_server.cc:307] RPC server started. Bound to: 127.29.194.2:42489
I20260807 15:11:22.719066 31037 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.194.2:42489 every 8 connection(s)
I20260807 15:11:22.719388 30846 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115480851507-30472-0/minicluster-data/ts-1/data/info.pb
I20260807 15:11:22.722729 30472 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7fZSIL/build/release/bin/kudu as pid 30846
I20260807 15:11:22.722811 30472 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115480851507-30472-0/minicluster-data/ts-1/wal/instance
I20260807 15:11:22.722981 30472 external_mini_cluster.cc:1428] Setting key c9d917f4fcc77fffc4c1488a18e84df8
Aug 07 15:11:22 dist-test-slave-42z9 krb5kdc[30481](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115482, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.194.2@KRBTEST.COM for kudu/127.29.194.62@KRBTEST.COM
I20260807 15:11:22.731194 30606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:22.720270 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.194.2:39427 (local address 127.29.194.62:45901)
0807 15:11:22.720407 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:22.720410 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:22.720608 (+ 198us) server_negotiation.cc:407] Connection header received
0807 15:11:22.721156 (+ 548us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:22.721157 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:22.721169 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:22.721233 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:11:22.722335 (+ 1102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:22.722969 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:22.724311 (+ 1342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:22.724543 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:22.724614 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:22.727288 (+ 2674us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:11:22.727289 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:11:22.727290 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:11:22.727298 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:11:22.728714 (+ 1416us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:22.729204 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:22.729206 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:22.729206 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:22.729275 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:22.730340 (+ 1065us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:22.730341 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:22.730342 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:22.730526 (+ 184us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:11:22.730604 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0807 15:11:22.730982 (+ 378us) server_negotiation.cc:299] Negotiation successful
0807 15:11:22.731089 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260807 15:11:22.731803 31041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:22.720183 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:45901 (local address 127.29.194.2:39427)
0807 15:11:22.720468 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0807 15:11:22.720513 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0807 15:11:22.721022 (+ 509us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:11:22.721302 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:11:22.721305 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:11:22.721977 (+ 672us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:11:22.722188 (+ 211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:22.722193 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:22.723118 (+ 925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:22.723120 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:22.724179 (+ 1059us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:22.724184 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:22.724643 (+ 459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:22.724644 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:22.724792 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:22.725361 (+ 569us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:11:22.725373 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:11:22.727170 (+ 1797us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:11:22.728791 (+ 1621us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:11:22.728800 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:11:22.728802 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:11:22.729119 (+ 317us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:11:22.729418 (+ 299us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:11:22.729419 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:11:22.729419 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:11:22.729603 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:11:22.730596 (+ 993us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:11:22.730603 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:11:22.730808 (+ 205us) client_negotiation.cc:769] Sending connection context
0807 15:11:22.731351 (+ 543us) client_negotiation.cc:240] Negotiation successful
0807 15:11:22.731563 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":237,"thread_start_us":190,"threads_started":1}
I20260807 15:11:22.732370 31039 heartbeater.cc:347] Connected to a master server at 127.29.194.62:45901
I20260807 15:11:22.732473 31039 heartbeater.cc:464] Registering TS with master...
I20260807 15:11:22.732681 31039 heartbeater.cc:511] Master 127.29.194.62:45901 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.29.194.3@KRBTEST.COM; defaulting to no policy
I20260807 15:11:22.733093 30552 ts_manager.cc:194] Registered new tserver with Master: 63cdd5e13a484c4bbcac173ec370e7ae (127.29.194.2:42489)
I20260807 15:11:22.733522 30552 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.194.2@KRBTEST.COM'} at 127.29.194.2:39427
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.194.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.194.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.3.keytab.
Entry for principal kudu/127.29.194.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.3.keytab.
I20260807 15:11:22.752831 30472 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7fZSIL/build/release/bin/kudu
/tmp/dist-test-task7fZSIL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115480851507-30472-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115480851507-30472-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115480851507-30472-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115480851507-30472-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.194.3:0
--local_ip_for_outbound_sockets=127.29.194.3
--webserver_interface=127.29.194.3
--webserver_port=0
--tserver_master_addrs=127.29.194.62:45901
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.3.keytab
--principal=kudu/127.29.194.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.194.20:45795
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115480851507-30472-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115480851507-30472-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115480851507-30472-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:11:22.840718 31045 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:11:22.840958 31045 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:11:22.840986 31045 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:11:22.842276 31045 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:11:22.842330 31045 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:11:22.842358 31045 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.194.3
I20260807 15:11:22.843911 31045 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.194.20:45795
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115480851507-30472-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115480851507-30472-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.3.keytab
--principal=kudu/127.29.194.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.194.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115480851507-30472-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.194.3
--webserver_port=0
--tserver_master_addrs=127.29.194.62:45901
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31045
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.194.3
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115480851507-30472-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:11:22.844149 31045 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:11:22.844337 31045 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:11:22.849651 31055 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:11:22.849609 31053 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:11:22.849757 31045 server_base.cc:1034] running on GCE node
W20260807 15:11:22.849611 31057 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:11:22.850041 31045 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:11:22.850303 31045 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:11:22.851467 31045 hybrid_clock.cc:648] HybridClock initialized: now 1786115482851454 us; error 30 us; skew 500 ppm
Aug 07 15:11:22 dist-test-slave-42z9 krb5kdc[30481](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115482, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.194.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:11:22.854090 31045 init.cc:377] Logged in from keytab as kudu/127.29.194.3@KRBTEST.COM (short username kudu)
I20260807 15:11:22.854564 31045 webserver.cc:460] Webserver started at http://127.29.194.3:44551/ using document root <none> and password file <none>
I20260807 15:11:22.854776 31045 fs_manager.cc:359] Metadata directory not provided
I20260807 15:11:22.854827 31045 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:11:22.854969 31045 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:11:22.856076 31045 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115480851507-30472-0/minicluster-data/ts-2/data/instance:
uuid: "14c0f51f8ee34173827dddbb82e3efb5"
format_stamp: "Formatted at 2026-08-07 15:11:22 on dist-test-slave-42z9"
server_key: "275e9e66e80d4d110c5530148e589f71"
server_key_iv: "d8e74e1cc6b3c1bc91f7e14cf74e1f47"
server_key_version: "encryptionkey@0"
I20260807 15:11:22.856428 31045 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115480851507-30472-0/minicluster-data/ts-2/wal/instance:
uuid: "14c0f51f8ee34173827dddbb82e3efb5"
format_stamp: "Formatted at 2026-08-07 15:11:22 on dist-test-slave-42z9"
server_key: "275e9e66e80d4d110c5530148e589f71"
server_key_iv: "d8e74e1cc6b3c1bc91f7e14cf74e1f47"
server_key_version: "encryptionkey@0"
I20260807 15:11:22.858168 31045 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20260807 15:11:22.859058 31064 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:11:22.859220 31045 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:11:22.859313 31045 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115480851507-30472-0/minicluster-data/ts-2/data,/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115480851507-30472-0/minicluster-data/ts-2/wal
uuid: "14c0f51f8ee34173827dddbb82e3efb5"
format_stamp: "Formatted at 2026-08-07 15:11:22 on dist-test-slave-42z9"
server_key: "275e9e66e80d4d110c5530148e589f71"
server_key_iv: "d8e74e1cc6b3c1bc91f7e14cf74e1f47"
server_key_version: "encryptionkey@0"
I20260807 15:11:22.859395 31045 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115480851507-30472-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115480851507-30472-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115480851507-30472-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:11:22.875037 31045 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:11:22.877357 31045 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:11:22.877538 31045 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:11:22.877795 31045 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:11:22.878132 31045 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:11:22.878170 31045 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:11:22.878201 31045 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:11:22.878225 31045 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:11:22.884968 31045 rpc_server.cc:307] RPC server started. Bound to: 127.29.194.3:45225
I20260807 15:11:22.885020 31225 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.194.3:45225 every 8 connection(s)
I20260807 15:11:22.885391 31045 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115480851507-30472-0/minicluster-data/ts-2/data/info.pb
I20260807 15:11:22.889060 30472 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7fZSIL/build/release/bin/kudu as pid 31045
I20260807 15:11:22.889151 30472 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115480851507-30472-0/minicluster-data/ts-2/wal/instance
I20260807 15:11:22.889294 30472 external_mini_cluster.cc:1428] Setting key 0d74b44cc227673b267f1a3ea472b55b
Aug 07 15:11:22 dist-test-slave-42z9 krb5kdc[30481](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115482, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.194.3@KRBTEST.COM for kudu/127.29.194.62@KRBTEST.COM
I20260807 15:11:22.897411 30606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:22.886341 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.194.3:34811 (local address 127.29.194.62:45901)
0807 15:11:22.886451 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:22.886453 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:22.886772 (+ 319us) server_negotiation.cc:407] Connection header received
0807 15:11:22.887328 (+ 556us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:22.887329 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:22.887362 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:22.887420 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:11:22.888461 (+ 1041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:22.888970 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:22.890591 (+ 1621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:22.890847 (+ 256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:22.890914 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:22.893911 (+ 2997us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:11:22.893912 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:11:22.893912 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:11:22.893921 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:11:22.895495 (+ 1574us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:22.896095 (+ 600us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:22.896098 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:22.896099 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:22.896241 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:22.896642 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:22.896643 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:22.896644 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:22.896804 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:11:22.896885 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0807 15:11:22.897260 (+ 375us) server_negotiation.cc:299] Negotiation successful
0807 15:11:22.897341 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260807 15:11:22.897814 31229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:22.886308 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:45901 (local address 127.29.194.3:34811)
0807 15:11:22.886629 (+ 321us) negotiation.cc:107] Waiting for socket to connect
0807 15:11:22.886676 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0807 15:11:22.887214 (+ 538us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:11:22.887513 (+ 299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:11:22.887517 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:11:22.888139 (+ 622us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:11:22.888357 (+ 218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:22.888362 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:22.889062 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:22.889063 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:22.890508 (+ 1445us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:22.890512 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:22.890927 (+ 415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:22.890928 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:22.891076 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:22.891700 (+ 624us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:11:22.891710 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:11:22.893773 (+ 2063us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:11:22.895622 (+ 1849us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:11:22.895633 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:11:22.895635 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:11:22.895956 (+ 321us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:11:22.896353 (+ 397us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:11:22.896355 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:11:22.896355 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:11:22.896535 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:11:22.896917 (+ 382us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:11:22.896926 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:11:22.897114 (+ 188us) client_negotiation.cc:769] Sending connection context
0807 15:11:22.897240 (+ 126us) client_negotiation.cc:240] Negotiation successful
0807 15:11:22.897490 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":265,"thread_start_us":164,"threads_started":1}
I20260807 15:11:22.898530 31226 heartbeater.cc:347] Connected to a master server at 127.29.194.62:45901
I20260807 15:11:22.898656 31226 heartbeater.cc:464] Registering TS with master...
I20260807 15:11:22.898849 31226 heartbeater.cc:511] Master 127.29.194.62:45901 requested a full tablet report, sending...
I20260807 15:11:22.899313 30552 ts_manager.cc:194] Registered new tserver with Master: 14c0f51f8ee34173827dddbb82e3efb5 (127.29.194.3:45225)
I20260807 15:11:22.899736 30552 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.194.3@KRBTEST.COM'} at 127.29.194.3:34811
I20260807 15:11:22.902555 30472 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 07 15:11:22 dist-test-slave-42z9 krb5kdc[30481](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115482, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 07 15:11:22 dist-test-slave-42z9 krb5kdc[30481](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115482, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.29.194.1@KRBTEST.COM
I20260807 15:11:22.930044 31237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:22.921384 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45876 (local address 127.29.194.1:33079)
0807 15:11:22.921611 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:22.921617 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:22.921630 (+ 13us) server_negotiation.cc:407] Connection header received
0807 15:11:22.921661 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:22.921663 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:22.921679 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:22.921752 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:11:22.922392 (+ 640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:22.923327 (+ 935us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:22.924218 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:22.924628 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:22.924705 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:22.927125 (+ 2420us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:11:22.927137 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:11:22.927137 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:11:22.927145 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:11:22.928679 (+ 1534us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:22.929112 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:22.929117 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:22.929117 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:22.929184 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:22.929381 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:22.929382 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:22.929383 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:22.929592 (+ 209us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:11:22.929740 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0807 15:11:22.929824 (+ 84us) server_negotiation.cc:299] Negotiation successful
0807 15:11:22.929929 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":193,"thread_start_us":141,"threads_started":1}
W20260807 15:11:22.930495 30699 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45876
Aug 07 15:11:22 dist-test-slave-42z9 krb5kdc[30481](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115482, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 07 15:11:22 dist-test-slave-42z9 krb5kdc[30481](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115482, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.194.1@KRBTEST.COM
I20260807 15:11:22.958186 31237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:22.949608 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45878 (local address 127.29.194.1:33079)
0807 15:11:22.949684 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:22.949686 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:22.949717 (+ 31us) server_negotiation.cc:407] Connection header received
0807 15:11:22.949796 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:22.949797 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:22.949809 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:22.949864 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:11:22.950763 (+ 899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:22.951299 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:22.952125 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:22.952342 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:22.952402 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:22.955038 (+ 2636us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:11:22.955042 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:11:22.955042 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:11:22.955055 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:11:22.956693 (+ 1638us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:22.957133 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:22.957134 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:22.957135 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:22.957207 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:22.957413 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:22.957414 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:22.957415 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:22.957711 (+ 296us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:11:22.957840 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0807 15:11:22.957991 (+ 151us) server_negotiation.cc:299] Negotiation successful
0807 15:11:22.958092 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260807 15:11:22.958979 30472 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7fZSIL/build/release/bin/kudu with pid 30638
I20260807 15:11:22.974386 30472 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7fZSIL/build/release/bin/kudu with pid 30846
I20260807 15:11:22.991425 30472 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7fZSIL/build/release/bin/kudu with pid 31045
I20260807 15:11:23.001935 30472 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7fZSIL/build/release/bin/kudu with pid 30500
2026-08-07T15:11:23Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (2165 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115480851507-30472-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 15:11:23 dist-test-slave-42z9 krb5kdc[31245](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 15:11:23 dist-test-slave-42z9 krb5kdc[31245](info): set up 2 sockets
Aug 07 15:11:23 dist-test-slave-42z9 krb5kdc[31245](info): commencing operation
krb5kdc: starting...
W20260807 15:11:24.161127 30472 mini_kdc.cc:121] Time spent starting KDC: real 1.130s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115480851507-30472-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115480851507-30472-0/krb5kdc/test-user.keytab.
Aug 07 15:11:24 dist-test-slave-42z9 krb5kdc[31245](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115484, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T15:11:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T15:11:24Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.194.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.194.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.194.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.62.keytab.
Entry for principal kudu/127.29.194.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.62.keytab.
I20260807 15:11:24.296360 30472 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7fZSIL/build/release/bin/kudu
/tmp/dist-test-task7fZSIL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115480851507-30472-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115480851507-30472-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115480851507-30472-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115480851507-30472-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.194.62:42305
--webserver_interface=127.29.194.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.62.keytab
--principal=kudu/127.29.194.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.194.20:42521
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.194.62:42305
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115480851507-30472-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115480851507-30472-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115480851507-30472-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:11:24.383877 31260 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:11:24.384258 31260 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:11:24.384302 31260 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:11:24.386076 31260 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:11:24.386155 31260 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:11:24.386337 31260 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:11:24.386406 31260 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:11:24.386425 31260 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:11:24.389266 31260 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.194.20:42521
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115480851507-30472-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115480851507-30472-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.194.62:42305
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.62.keytab
--principal=kudu/127.29.194.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.194.62:42305
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115480851507-30472-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.194.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31260
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115480851507-30472-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:11:24.389628 31260 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:11:24.389868 31260 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:11:24.395233 31271 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:11:24.395248 31266 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:11:24.395233 31268 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:11:24.395865 31260 server_base.cc:1034] running on GCE node
I20260807 15:11:24.396127 31260 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:11:24.396442 31260 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:11:24.397620 31260 hybrid_clock.cc:648] HybridClock initialized: now 1786115484397603 us; error 30 us; skew 500 ppm
Aug 07 15:11:24 dist-test-slave-42z9 krb5kdc[31245](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115484, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.194.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:11:24.401347 31260 init.cc:377] Logged in from keytab as kudu/127.29.194.62@KRBTEST.COM (short username kudu)
I20260807 15:11:24.401953 31260 webserver.cc:460] Webserver started at http://127.29.194.62:32775/ using document root <none> and password file <none>
I20260807 15:11:24.402185 31260 fs_manager.cc:359] Metadata directory not provided
I20260807 15:11:24.402240 31260 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:11:24.402382 31260 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:11:24.403595 31260 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115480851507-30472-0/minicluster-data/master-0/data/instance:
uuid: "d69ee384fa2c40f4b972a1610c966e38"
format_stamp: "Formatted at 2026-08-07 15:11:24 on dist-test-slave-42z9"
server_key: "be171fc09baed392ede3cb3349508a41"
server_key_iv: "283418ac7ee3300e950ad36ebe609b95"
server_key_version: "encryptionkey@0"
I20260807 15:11:24.404093 31260 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115480851507-30472-0/minicluster-data/master-0/wal/instance:
uuid: "d69ee384fa2c40f4b972a1610c966e38"
format_stamp: "Formatted at 2026-08-07 15:11:24 on dist-test-slave-42z9"
server_key: "be171fc09baed392ede3cb3349508a41"
server_key_iv: "283418ac7ee3300e950ad36ebe609b95"
server_key_version: "encryptionkey@0"
I20260807 15:11:24.405584 31260 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 15:11:24.406418 31283 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:11:24.406597 31260 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:11:24.406687 31260 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115480851507-30472-0/minicluster-data/master-0/data,/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115480851507-30472-0/minicluster-data/master-0/wal
uuid: "d69ee384fa2c40f4b972a1610c966e38"
format_stamp: "Formatted at 2026-08-07 15:11:24 on dist-test-slave-42z9"
server_key: "be171fc09baed392ede3cb3349508a41"
server_key_iv: "283418ac7ee3300e950ad36ebe609b95"
server_key_version: "encryptionkey@0"
I20260807 15:11:24.406751 31260 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115480851507-30472-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115480851507-30472-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115480851507-30472-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:11:24.424937 31260 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:11:24.428260 31260 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:11:24.428424 31260 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:11:24.433131 31260 rpc_server.cc:307] RPC server started. Bound to: 127.29.194.62:42305
I20260807 15:11:24.433244 31355 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.194.62:42305 every 8 connection(s)
I20260807 15:11:24.433624 31260 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115480851507-30472-0/minicluster-data/master-0/data/info.pb
I20260807 15:11:24.433808 30472 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7fZSIL/build/release/bin/kudu as pid 31260
I20260807 15:11:24.433910 30472 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115480851507-30472-0/minicluster-data/master-0/wal/instance
I20260807 15:11:24.434099 30472 external_mini_cluster.cc:1428] Setting key 943d35eab184f9b8c7c9e119637aa06b
I20260807 15:11:24.435472 31356 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:11:24.439746 31356 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d69ee384fa2c40f4b972a1610c966e38: Bootstrap starting.
I20260807 15:11:24.440549 31356 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d69ee384fa2c40f4b972a1610c966e38: Neither blocks nor log segments found. Creating new log.
I20260807 15:11:24.440915 31356 log.cc:824] T 00000000000000000000000000000000 P d69ee384fa2c40f4b972a1610c966e38: Log is configured to *not* fsync() on all Append() calls
I20260807 15:11:24.442162 31356 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d69ee384fa2c40f4b972a1610c966e38: No bootstrap required, opened a new log
Aug 07 15:11:24 dist-test-slave-42z9 krb5kdc[31245](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115484, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.194.62@KRBTEST.COM
I20260807 15:11:24.444878 31356 raft_consensus.cc:348] T 00000000000000000000000000000000 P d69ee384fa2c40f4b972a1610c966e38 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d69ee384fa2c40f4b972a1610c966e38" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 42305 } }
I20260807 15:11:24.445052 31356 raft_consensus.cc:374] T 00000000000000000000000000000000 P d69ee384fa2c40f4b972a1610c966e38 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:11:24.445080 31356 raft_consensus.cc:729] T 00000000000000000000000000000000 P d69ee384fa2c40f4b972a1610c966e38 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d69ee384fa2c40f4b972a1610c966e38, State: Initialized, Role: FOLLOWER
I20260807 15:11:24.445202 31356 consensus_queue.cc:260] T 00000000000000000000000000000000 P d69ee384fa2c40f4b972a1610c966e38 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d69ee384fa2c40f4b972a1610c966e38" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 42305 } }
I20260807 15:11:24.445289 31356 raft_consensus.cc:388] T 00000000000000000000000000000000 P d69ee384fa2c40f4b972a1610c966e38 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:11:24.445323 31356 raft_consensus.cc:482] T 00000000000000000000000000000000 P d69ee384fa2c40f4b972a1610c966e38 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:11:24.445384 31356 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d69ee384fa2c40f4b972a1610c966e38 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:11:24.446257 31356 raft_consensus.cc:504] T 00000000000000000000000000000000 P d69ee384fa2c40f4b972a1610c966e38 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d69ee384fa2c40f4b972a1610c966e38" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 42305 } }
I20260807 15:11:24.446388 31356 leader_election.cc:302] T 00000000000000000000000000000000 P d69ee384fa2c40f4b972a1610c966e38 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d69ee384fa2c40f4b972a1610c966e38; no voters:
I20260807 15:11:24.446596 31356 leader_election.cc:288] T 00000000000000000000000000000000 P d69ee384fa2c40f4b972a1610c966e38 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 15:11:24.446648 31362 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d69ee384fa2c40f4b972a1610c966e38 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:11:24.446820 31362 raft_consensus.cc:686] T 00000000000000000000000000000000 P d69ee384fa2c40f4b972a1610c966e38 [term 1 LEADER]: Becoming Leader. State: Replica: d69ee384fa2c40f4b972a1610c966e38, State: Running, Role: LEADER
I20260807 15:11:24.446894 31356 sys_catalog.cc:564] T 00000000000000000000000000000000 P d69ee384fa2c40f4b972a1610c966e38 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:11:24.446933 31362 consensus_queue.cc:237] T 00000000000000000000000000000000 P d69ee384fa2c40f4b972a1610c966e38 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d69ee384fa2c40f4b972a1610c966e38" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 42305 } }
I20260807 15:11:24.447642 31363 sys_catalog.cc:455] T 00000000000000000000000000000000 P d69ee384fa2c40f4b972a1610c966e38 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d69ee384fa2c40f4b972a1610c966e38" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d69ee384fa2c40f4b972a1610c966e38" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 42305 } } }
I20260807 15:11:24.447825 31363 sys_catalog.cc:458] T 00000000000000000000000000000000 P d69ee384fa2c40f4b972a1610c966e38 [sys.catalog]: This master's current role is: LEADER
I20260807 15:11:24.448062 31363 sys_catalog.cc:455] T 00000000000000000000000000000000 P d69ee384fa2c40f4b972a1610c966e38 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d69ee384fa2c40f4b972a1610c966e38. Latest consensus state: current_term: 1 leader_uuid: "d69ee384fa2c40f4b972a1610c966e38" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d69ee384fa2c40f4b972a1610c966e38" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 42305 } } }
I20260807 15:11:24.448146 31363 sys_catalog.cc:458] T 00000000000000000000000000000000 P d69ee384fa2c40f4b972a1610c966e38 [sys.catalog]: This master's current role is: LEADER
I20260807 15:11:24.448405 31359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:24.434939 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33470 (local address 127.29.194.62:42305)
0807 15:11:24.435178 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:24.435189 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:24.435221 (+ 32us) server_negotiation.cc:407] Connection header received
0807 15:11:24.435821 (+ 600us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:24.435829 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:24.435888 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:24.436052 (+ 164us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:11:24.436962 (+ 910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:24.438092 (+ 1130us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:24.439273 (+ 1181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:24.439840 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:24.439981 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:24.443400 (+ 3419us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:11:24.443405 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:11:24.443406 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:11:24.443423 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:11:24.445753 (+ 2330us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:24.446323 (+ 570us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:24.446328 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:24.446328 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:24.446482 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:24.446786 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:24.446787 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:24.446788 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:24.447322 (+ 534us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:11:24.447496 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0807 15:11:24.447888 (+ 392us) server_negotiation.cc:299] Negotiation successful
0807 15:11:24.448108 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":168,"thread_start_us":114,"threads_started":1}
I20260807 15:11:24.448308 31375 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:11:24.449699 31375 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:11:24.451795 31375 catalog_manager.cc:1347] Generated new cluster ID: efe98007d68048d6895cd17b9d54537d
I20260807 15:11:24.451867 31375 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:11:24.456530 31375 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:11:24.457221 31375 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:11:24.462787 31375 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d69ee384fa2c40f4b972a1610c966e38: Generated new TSK 0
I20260807 15:11:24.463011 31375 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.29.194.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.194.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.194.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.1.keytab.
Entry for principal kudu/127.29.194.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.1.keytab.
I20260807 15:11:24.498978 30472 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7fZSIL/build/release/bin/kudu
/tmp/dist-test-task7fZSIL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115480851507-30472-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115480851507-30472-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115480851507-30472-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115480851507-30472-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.194.1:0
--local_ip_for_outbound_sockets=127.29.194.1
--webserver_interface=127.29.194.1
--webserver_port=0
--tserver_master_addrs=127.29.194.62:42305
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.1.keytab
--principal=kudu/127.29.194.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.194.20:42521
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115480851507-30472-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115480851507-30472-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115480851507-30472-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:11:24.591707 31388 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:11:24.591933 31388 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:11:24.591960 31388 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:11:24.593204 31388 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:11:24.593257 31388 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:11:24.593286 31388 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.194.1
I20260807 15:11:24.594988 31388 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.194.20:42521
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115480851507-30472-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115480851507-30472-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.1.keytab
--principal=kudu/127.29.194.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.194.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115480851507-30472-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.194.1
--webserver_port=0
--tserver_master_addrs=127.29.194.62:42305
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31388
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.194.1
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115480851507-30472-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:11:24.595256 31388 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:11:24.595502 31388 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:11:24.601236 31395 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:11:24.601253 31398 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:11:24.601331 31396 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:11:24.601315 31388 server_base.cc:1034] running on GCE node
I20260807 15:11:24.601926 31388 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:11:24.602216 31388 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:11:24.603390 31388 hybrid_clock.cc:648] HybridClock initialized: now 1786115484603373 us; error 37 us; skew 500 ppm
Aug 07 15:11:24 dist-test-slave-42z9 krb5kdc[31245](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115484, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.194.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:11:24.606472 31388 init.cc:377] Logged in from keytab as kudu/127.29.194.1@KRBTEST.COM (short username kudu)
I20260807 15:11:24.606981 31388 webserver.cc:460] Webserver started at http://127.29.194.1:33067/ using document root <none> and password file <none>
I20260807 15:11:24.607228 31388 fs_manager.cc:359] Metadata directory not provided
I20260807 15:11:24.607280 31388 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:11:24.607409 31388 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:11:24.608486 31388 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115480851507-30472-0/minicluster-data/ts-0/data/instance:
uuid: "8dff9d4f4e624e2186a57a2085f236d3"
format_stamp: "Formatted at 2026-08-07 15:11:24 on dist-test-slave-42z9"
server_key: "c3e2e415b256e981a69739693f4301bf"
server_key_iv: "cfaee6088a3367a2b8a4b0dc0ab65e18"
server_key_version: "encryptionkey@0"
I20260807 15:11:24.608824 31388 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115480851507-30472-0/minicluster-data/ts-0/wal/instance:
uuid: "8dff9d4f4e624e2186a57a2085f236d3"
format_stamp: "Formatted at 2026-08-07 15:11:24 on dist-test-slave-42z9"
server_key: "c3e2e415b256e981a69739693f4301bf"
server_key_iv: "cfaee6088a3367a2b8a4b0dc0ab65e18"
server_key_version: "encryptionkey@0"
I20260807 15:11:24.610312 31388 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260807 15:11:24.611312 31408 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:11:24.611626 31388 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 15:11:24.611884 31388 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115480851507-30472-0/minicluster-data/ts-0/data,/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115480851507-30472-0/minicluster-data/ts-0/wal
uuid: "8dff9d4f4e624e2186a57a2085f236d3"
format_stamp: "Formatted at 2026-08-07 15:11:24 on dist-test-slave-42z9"
server_key: "c3e2e415b256e981a69739693f4301bf"
server_key_iv: "cfaee6088a3367a2b8a4b0dc0ab65e18"
server_key_version: "encryptionkey@0"
I20260807 15:11:24.611976 31388 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115480851507-30472-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115480851507-30472-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115480851507-30472-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:11:24.628998 31388 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:11:24.632189 31388 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:11:24.632355 31388 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:11:24.632609 31388 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:11:24.632928 31388 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:11:24.632961 31388 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:11:24.632992 31388 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:11:24.633010 31388 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:11:24.639063 31388 rpc_server.cc:307] RPC server started. Bound to: 127.29.194.1:45001
I20260807 15:11:24.639274 31577 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.194.1:45001 every 8 connection(s)
I20260807 15:11:24.639533 31388 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115480851507-30472-0/minicluster-data/ts-0/data/info.pb
I20260807 15:11:24.645967 30472 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7fZSIL/build/release/bin/kudu as pid 31388
I20260807 15:11:24.646081 30472 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115480851507-30472-0/minicluster-data/ts-0/wal/instance
I20260807 15:11:24.646245 30472 external_mini_cluster.cc:1428] Setting key e9c8ce3f987cc3ab8cbd134315692b95
Aug 07 15:11:24 dist-test-slave-42z9 krb5kdc[31245](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115484, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.194.1@KRBTEST.COM for kudu/127.29.194.62@KRBTEST.COM
I20260807 15:11:24.651098 31359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:24.640534 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.194.1:42891 (local address 127.29.194.62:42305)
0807 15:11:24.640631 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:24.640633 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:24.640981 (+ 348us) server_negotiation.cc:407] Connection header received
0807 15:11:24.642161 (+ 1180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:24.642163 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:24.642178 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:24.642227 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:11:24.643004 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:24.643527 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:24.644581 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:24.644787 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:24.644845 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:24.648170 (+ 3325us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:11:24.648172 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:11:24.648172 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:11:24.648181 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:11:24.649601 (+ 1420us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:24.650118 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:24.650120 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:24.650120 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:24.650191 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:24.650424 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:24.650425 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:24.650425 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:24.650576 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:11:24.650648 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0807 15:11:24.650896 (+ 248us) server_negotiation.cc:299] Negotiation successful
0807 15:11:24.650991 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260807 15:11:24.651196 31580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:24.640386 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:42305 (local address 127.29.194.1:42891)
0807 15:11:24.640657 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0807 15:11:24.640730 (+ 73us) client_negotiation.cc:174] Beginning negotiation
0807 15:11:24.642025 (+ 1295us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:11:24.642318 (+ 293us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:11:24.642321 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:11:24.642670 (+ 349us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:11:24.642896 (+ 226us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:24.642900 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:24.643626 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:24.643627 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:24.644477 (+ 850us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:24.644479 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:24.644871 (+ 392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:24.644872 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:24.645001 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:24.645620 (+ 619us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:11:24.645629 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:11:24.648074 (+ 2445us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:11:24.649683 (+ 1609us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:11:24.649693 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:11:24.649695 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:11:24.650010 (+ 315us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:11:24.650261 (+ 251us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:11:24.650261 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:11:24.650262 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:11:24.650367 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:11:24.650639 (+ 272us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:11:24.650643 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:11:24.650773 (+ 130us) client_negotiation.cc:769] Sending connection context
0807 15:11:24.650856 (+ 83us) client_negotiation.cc:240] Negotiation successful
0807 15:11:24.651010 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":99,"threads_started":1}
I20260807 15:11:24.651732 31578 heartbeater.cc:347] Connected to a master server at 127.29.194.62:42305
I20260807 15:11:24.651844 31578 heartbeater.cc:464] Registering TS with master...
I20260807 15:11:24.652036 31578 heartbeater.cc:511] Master 127.29.194.62:42305 requested a full tablet report, sending...
I20260807 15:11:24.652712 31309 ts_manager.cc:194] Registered new tserver with Master: 8dff9d4f4e624e2186a57a2085f236d3 (127.29.194.1:45001)
I20260807 15:11:24.653667 31309 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.194.1@KRBTEST.COM'} at 127.29.194.1:42891
WARNING: no policy specified for kudu/127.29.194.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.194.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.194.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.2.keytab.
Entry for principal kudu/127.29.194.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.2.keytab.
I20260807 15:11:24.677157 30472 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7fZSIL/build/release/bin/kudu
/tmp/dist-test-task7fZSIL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115480851507-30472-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115480851507-30472-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115480851507-30472-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115480851507-30472-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.194.2:0
--local_ip_for_outbound_sockets=127.29.194.2
--webserver_interface=127.29.194.2
--webserver_port=0
--tserver_master_addrs=127.29.194.62:42305
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.2.keytab
--principal=kudu/127.29.194.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.194.20:42521
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115480851507-30472-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115480851507-30472-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115480851507-30472-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:11:24.767549 31586 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:11:24.767800 31586 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:11:24.767836 31586 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:11:24.769109 31586 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:11:24.769171 31586 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:11:24.769210 31586 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.194.2
I20260807 15:11:24.771108 31586 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.194.20:42521
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115480851507-30472-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115480851507-30472-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.2.keytab
--principal=kudu/127.29.194.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.194.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115480851507-30472-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.194.2
--webserver_port=0
--tserver_master_addrs=127.29.194.62:42305
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31586
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.194.2
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115480851507-30472-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:11:24.771420 31586 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:11:24.771682 31586 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:11:24.777261 31594 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:11:24.778772 31593 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:11:24.778787 31597 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:11:24.778805 31586 server_base.cc:1034] running on GCE node
I20260807 15:11:24.779155 31586 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:11:24.779448 31586 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:11:24.780705 31586 hybrid_clock.cc:648] HybridClock initialized: now 1786115484780656 us; error 52 us; skew 500 ppm
Aug 07 15:11:24 dist-test-slave-42z9 krb5kdc[31245](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115484, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.194.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:11:24.784301 31586 init.cc:377] Logged in from keytab as kudu/127.29.194.2@KRBTEST.COM (short username kudu)
I20260807 15:11:24.784880 31586 webserver.cc:460] Webserver started at http://127.29.194.2:39409/ using document root <none> and password file <none>
I20260807 15:11:24.785117 31586 fs_manager.cc:359] Metadata directory not provided
I20260807 15:11:24.785172 31586 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:11:24.785301 31586 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:11:24.786478 31586 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115480851507-30472-0/minicluster-data/ts-1/data/instance:
uuid: "d20885f827b64f90a4004ff81c309ae0"
format_stamp: "Formatted at 2026-08-07 15:11:24 on dist-test-slave-42z9"
server_key: "36c27ddf44c38cf9d431bb091f9d3156"
server_key_iv: "624d0a5a45290b2c838656f384b97580"
server_key_version: "encryptionkey@0"
I20260807 15:11:24.786845 31586 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115480851507-30472-0/minicluster-data/ts-1/wal/instance:
uuid: "d20885f827b64f90a4004ff81c309ae0"
format_stamp: "Formatted at 2026-08-07 15:11:24 on dist-test-slave-42z9"
server_key: "36c27ddf44c38cf9d431bb091f9d3156"
server_key_iv: "624d0a5a45290b2c838656f384b97580"
server_key_version: "encryptionkey@0"
I20260807 15:11:24.788309 31586 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 15:11:24.789330 31608 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:11:24.789562 31586 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:11:24.789664 31586 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115480851507-30472-0/minicluster-data/ts-1/data,/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115480851507-30472-0/minicluster-data/ts-1/wal
uuid: "d20885f827b64f90a4004ff81c309ae0"
format_stamp: "Formatted at 2026-08-07 15:11:24 on dist-test-slave-42z9"
server_key: "36c27ddf44c38cf9d431bb091f9d3156"
server_key_iv: "624d0a5a45290b2c838656f384b97580"
server_key_version: "encryptionkey@0"
I20260807 15:11:24.789738 31586 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115480851507-30472-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115480851507-30472-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115480851507-30472-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:11:24.813694 31586 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:11:24.816049 31586 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:11:24.816210 31586 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:11:24.816452 31586 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:11:24.816785 31586 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:11:24.816821 31586 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:11:24.816852 31586 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:11:24.816876 31586 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:11:24.823478 31586 rpc_server.cc:307] RPC server started. Bound to: 127.29.194.2:34407
I20260807 15:11:24.823534 31781 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.194.2:34407 every 8 connection(s)
I20260807 15:11:24.823886 31586 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115480851507-30472-0/minicluster-data/ts-1/data/info.pb
I20260807 15:11:24.824854 30472 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7fZSIL/build/release/bin/kudu as pid 31586
I20260807 15:11:24.824941 30472 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115480851507-30472-0/minicluster-data/ts-1/wal/instance
I20260807 15:11:24.825121 30472 external_mini_cluster.cc:1428] Setting key 1ce857f56ee9a6d3fe1b912335b71b7c
Aug 07 15:11:24 dist-test-slave-42z9 krb5kdc[31245](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115484, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.194.2@KRBTEST.COM for kudu/127.29.194.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.29.194.3@KRBTEST.COM; defaulting to no policy
I20260807 15:11:24.835255 31359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:24.824747 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.194.2:36939 (local address 127.29.194.62:42305)
0807 15:11:24.824817 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:24.824819 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:24.825034 (+ 215us) server_negotiation.cc:407] Connection header received
0807 15:11:24.826186 (+ 1152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:24.826187 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:24.826200 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:24.826265 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:11:24.826897 (+ 632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:24.827579 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:24.829088 (+ 1509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:24.829285 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:24.829349 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:24.832211 (+ 2862us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:11:24.832213 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:11:24.832213 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:11:24.832221 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:11:24.833590 (+ 1369us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:24.834165 (+ 575us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:24.834166 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:24.834167 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:24.834236 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:24.834535 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:24.834536 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:24.834536 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:24.834688 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:11:24.834759 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0807 15:11:24.835057 (+ 298us) server_negotiation.cc:299] Negotiation successful
0807 15:11:24.835157 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260807 15:11:24.835457 31787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:24.824696 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:42305 (local address 127.29.194.2:36939)
0807 15:11:24.824900 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0807 15:11:24.824944 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0807 15:11:24.826083 (+ 1139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:11:24.826328 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:11:24.826331 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:11:24.826612 (+ 281us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:11:24.826815 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:24.826819 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:24.827703 (+ 884us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:24.827705 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:24.828973 (+ 1268us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:24.828977 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:24.829384 (+ 407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:24.829386 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:24.829552 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:24.830177 (+ 625us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:11:24.830188 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:11:24.832099 (+ 1911us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:11:24.833679 (+ 1580us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:11:24.833689 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:11:24.833692 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:11:24.834033 (+ 341us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:11:24.834318 (+ 285us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:11:24.834319 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:11:24.834320 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:11:24.834469 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:11:24.834763 (+ 294us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:11:24.834773 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:11:24.834922 (+ 149us) client_negotiation.cc:769] Sending connection context
0807 15:11:24.835026 (+ 104us) client_negotiation.cc:240] Negotiation successful
0807 15:11:24.835223 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":91,"threads_started":1}
I20260807 15:11:24.836314 31784 heartbeater.cc:347] Connected to a master server at 127.29.194.62:42305
I20260807 15:11:24.836421 31784 heartbeater.cc:464] Registering TS with master...
I20260807 15:11:24.836628 31784 heartbeater.cc:511] Master 127.29.194.62:42305 requested a full tablet report, sending...
I20260807 15:11:24.837253 31309 ts_manager.cc:194] Registered new tserver with Master: d20885f827b64f90a4004ff81c309ae0 (127.29.194.2:34407)
I20260807 15:11:24.837702 31309 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.194.2@KRBTEST.COM'} at 127.29.194.2:36939
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.194.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.194.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.3.keytab.
Entry for principal kudu/127.29.194.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.3.keytab.
I20260807 15:11:24.857913 30472 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7fZSIL/build/release/bin/kudu
/tmp/dist-test-task7fZSIL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115480851507-30472-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115480851507-30472-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115480851507-30472-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115480851507-30472-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.194.3:0
--local_ip_for_outbound_sockets=127.29.194.3
--webserver_interface=127.29.194.3
--webserver_port=0
--tserver_master_addrs=127.29.194.62:42305
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.3.keytab
--principal=kudu/127.29.194.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.194.20:42521
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115480851507-30472-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115480851507-30472-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115480851507-30472-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:11:24.943120 31790 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:11:24.943365 31790 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:11:24.943401 31790 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:11:24.944696 31790 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:11:24.944751 31790 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:11:24.944790 31790 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.194.3
I20260807 15:11:24.946619 31790 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.194.20:42521
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115480851507-30472-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115480851507-30472-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.3.keytab
--principal=kudu/127.29.194.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.194.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115480851507-30472-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.194.3
--webserver_port=0
--tserver_master_addrs=127.29.194.62:42305
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31790
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.194.3
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115480851507-30472-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:11:24.946902 31790 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:11:24.947139 31790 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:11:24.953042 31797 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:11:24.953022 31796 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:11:24.953022 31801 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:11:24.953549 31790 server_base.cc:1034] running on GCE node
I20260807 15:11:24.953886 31790 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:11:24.954237 31790 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:11:24.955438 31790 hybrid_clock.cc:648] HybridClock initialized: now 1786115484955411 us; error 41 us; skew 500 ppm
Aug 07 15:11:24 dist-test-slave-42z9 krb5kdc[31245](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115484, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.194.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:11:24.959115 31790 init.cc:377] Logged in from keytab as kudu/127.29.194.3@KRBTEST.COM (short username kudu)
I20260807 15:11:24.959707 31790 webserver.cc:460] Webserver started at http://127.29.194.3:35735/ using document root <none> and password file <none>
I20260807 15:11:24.959936 31790 fs_manager.cc:359] Metadata directory not provided
I20260807 15:11:24.959987 31790 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:11:24.960126 31790 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:11:24.961277 31790 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115480851507-30472-0/minicluster-data/ts-2/data/instance:
uuid: "9cde9d7f2650466499b01c1b3f53bf69"
format_stamp: "Formatted at 2026-08-07 15:11:24 on dist-test-slave-42z9"
server_key: "a33d55c946c4d9ef0bf16f0edfac32d8"
server_key_iv: "383df8a73e390bd8e79ad2bf3616e7ca"
server_key_version: "encryptionkey@0"
I20260807 15:11:24.961784 31790 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115480851507-30472-0/minicluster-data/ts-2/wal/instance:
uuid: "9cde9d7f2650466499b01c1b3f53bf69"
format_stamp: "Formatted at 2026-08-07 15:11:24 on dist-test-slave-42z9"
server_key: "a33d55c946c4d9ef0bf16f0edfac32d8"
server_key_iv: "383df8a73e390bd8e79ad2bf3616e7ca"
server_key_version: "encryptionkey@0"
I20260807 15:11:24.963430 31790 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 15:11:24.964612 31810 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:11:24.964854 31790 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:11:24.964965 31790 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115480851507-30472-0/minicluster-data/ts-2/data,/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115480851507-30472-0/minicluster-data/ts-2/wal
uuid: "9cde9d7f2650466499b01c1b3f53bf69"
format_stamp: "Formatted at 2026-08-07 15:11:24 on dist-test-slave-42z9"
server_key: "a33d55c946c4d9ef0bf16f0edfac32d8"
server_key_iv: "383df8a73e390bd8e79ad2bf3616e7ca"
server_key_version: "encryptionkey@0"
I20260807 15:11:24.965044 31790 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115480851507-30472-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115480851507-30472-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115480851507-30472-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:11:24.988205 31790 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:11:24.990455 31790 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:11:24.990612 31790 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:11:24.990859 31790 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:11:24.991206 31790 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:11:24.991251 31790 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:11:24.991283 31790 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:11:24.991304 31790 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:11:24.998049 31790 rpc_server.cc:307] RPC server started. Bound to: 127.29.194.3:39825
I20260807 15:11:24.998162 31979 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.194.3:39825 every 8 connection(s)
I20260807 15:11:24.998468 31790 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115480851507-30472-0/minicluster-data/ts-2/data/info.pb
I20260807 15:11:25.004988 30472 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7fZSIL/build/release/bin/kudu as pid 31790
I20260807 15:11:25.005112 30472 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115480851507-30472-0/minicluster-data/ts-2/wal/instance
I20260807 15:11:25.005272 30472 external_mini_cluster.cc:1428] Setting key 89177fe36ceef3c521db4524f58618f2
Aug 07 15:11:25 dist-test-slave-42z9 krb5kdc[31245](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115484, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.194.3@KRBTEST.COM for kudu/127.29.194.62@KRBTEST.COM
I20260807 15:11:25.011258 31359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:24.999641 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.194.3:42633 (local address 127.29.194.62:42305)
0807 15:11:24.999764 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:24.999767 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:25.000154 (+ 387us) server_negotiation.cc:407] Connection header received
0807 15:11:25.001136 (+ 982us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:25.001138 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:25.001151 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:25.001232 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:11:25.002136 (+ 904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:25.002766 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:25.004412 (+ 1646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:25.004647 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:25.004732 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:25.007641 (+ 2909us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:11:25.007642 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:11:25.007643 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:11:25.007651 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:11:25.009009 (+ 1358us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:25.009872 (+ 863us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:25.009873 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:25.009873 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:25.009940 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:25.010347 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:25.010350 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:25.010350 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:25.010644 (+ 294us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:11:25.010710 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0807 15:11:25.011086 (+ 376us) server_negotiation.cc:299] Negotiation successful
0807 15:11:25.011183 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260807 15:11:25.011386 31983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:24.999598 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:42305 (local address 127.29.194.3:42633)
0807 15:11:24.999847 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0807 15:11:24.999903 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0807 15:11:25.000963 (+ 1060us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:11:25.001327 (+ 364us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:11:25.001331 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:11:25.001704 (+ 373us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:11:25.001920 (+ 216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:25.001925 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:25.002998 (+ 1073us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:25.003000 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:25.004203 (+ 1203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:25.004210 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:25.004770 (+ 560us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:25.004772 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:25.004955 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:25.005594 (+ 639us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:11:25.005606 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:11:25.007510 (+ 1904us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:11:25.009116 (+ 1606us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:11:25.009125 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:11:25.009127 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:11:25.009700 (+ 573us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:11:25.010045 (+ 345us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:11:25.010047 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:11:25.010047 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:11:25.010200 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:11:25.010742 (+ 542us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:11:25.010749 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:11:25.010908 (+ 159us) client_negotiation.cc:769] Sending connection context
0807 15:11:25.011012 (+ 104us) client_negotiation.cc:240] Negotiation successful
0807 15:11:25.011185 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":141,"threads_started":1}
I20260807 15:11:25.011982 31981 heartbeater.cc:347] Connected to a master server at 127.29.194.62:42305
I20260807 15:11:25.012108 31981 heartbeater.cc:464] Registering TS with master...
I20260807 15:11:25.012315 31981 heartbeater.cc:511] Master 127.29.194.62:42305 requested a full tablet report, sending...
I20260807 15:11:25.012728 31309 ts_manager.cc:194] Registered new tserver with Master: 9cde9d7f2650466499b01c1b3f53bf69 (127.29.194.3:39825)
I20260807 15:11:25.013216 31309 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.194.3@KRBTEST.COM'} at 127.29.194.3:42633
I20260807 15:11:25.017869 30472 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 15:11:25.026600 31359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:25.019677 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33480 (local address 127.29.194.62:42305)
0807 15:11:25.019772 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:25.019774 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:25.020022 (+ 248us) server_negotiation.cc:407] Connection header received
0807 15:11:25.020055 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:25.020056 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:25.020067 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:25.020123 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:11:25.020901 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:25.021538 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:25.022405 (+ 867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:25.022686 (+ 281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:25.022758 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:25.024110 (+ 1352us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:11:25.024112 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:11:25.024113 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:11:25.024123 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:11:25.025501 (+ 1378us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:25.025841 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:25.025841 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:25.025842 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:25.025899 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:25.026073 (+ 174us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:25.026074 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:25.026074 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:25.026239 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:11:25.026315 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0807 15:11:25.026418 (+ 103us) server_negotiation.cc:299] Negotiation successful
0807 15:11:25.026491 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260807 15:11:25.029512 31309 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33480:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 15:11:25.031519 31309 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 15:11:25.039665 31997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:25.034810 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.2:34407 (local address 127.0.0.1:48420)
0807 15:11:25.035167 (+ 357us) negotiation.cc:107] Waiting for socket to connect
0807 15:11:25.035236 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0807 15:11:25.035347 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:11:25.035572 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:11:25.035576 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:11:25.035579 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:11:25.035680 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:25.035683 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:25.037119 (+ 1436us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:25.037120 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:25.038369 (+ 1249us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:25.038372 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:25.039309 (+ 937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:25.039310 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:25.039418 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:25.039435 (+ 17us) client_negotiation.cc:769] Sending connection context
0807 15:11:25.039504 (+ 69us) client_negotiation.cc:240] Negotiation successful
0807 15:11:25.039579 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":323,"thread_start_us":165,"threads_started":1}
I20260807 15:11:25.039677 32002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:25.035101 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48420 (local address 127.29.194.2:34407)
0807 15:11:25.035353 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:25.035360 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:25.035374 (+ 14us) server_negotiation.cc:407] Connection header received
0807 15:11:25.035436 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:25.035439 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:25.035458 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:25.035544 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:11:25.035761 (+ 217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:25.036966 (+ 1205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:25.038487 (+ 1521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:25.039225 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:25.039297 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:25.039379 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0807 15:11:25.039529 (+ 150us) server_negotiation.cc:299] Negotiation successful
0807 15:11:25.039579 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":213,"thread_start_us":112,"threads_started":1}
I20260807 15:11:25.040079 31996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:25.034721 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.3:39825 (local address 127.0.0.1:53636)
0807 15:11:25.035147 (+ 426us) negotiation.cc:107] Waiting for socket to connect
0807 15:11:25.035192 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0807 15:11:25.035267 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:11:25.036029 (+ 762us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:11:25.036030 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:11:25.036033 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:11:25.036101 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:25.036103 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:25.037619 (+ 1516us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:25.037620 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:25.038644 (+ 1024us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:25.038646 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:25.039661 (+ 1015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:25.039662 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:25.039718 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:25.039723 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 15:11:25.039916 (+ 193us) client_negotiation.cc:240] Negotiation successful
0807 15:11:25.039989 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":395,"thread_start_us":83,"threads_started":1}
I20260807 15:11:25.040395 31998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:25.034859 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53636 (local address 127.29.194.3:39825)
0807 15:11:25.035809 (+ 950us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:25.035814 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:25.035824 (+ 10us) server_negotiation.cc:407] Connection header received
0807 15:11:25.035859 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:25.035861 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:25.035877 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:25.035945 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:11:25.036206 (+ 261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:25.037491 (+ 1285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:25.038737 (+ 1246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:25.039506 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:25.039575 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:25.039630 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0807 15:11:25.040263 (+ 633us) server_negotiation.cc:299] Negotiation successful
0807 15:11:25.040313 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":910,"thread_start_us":124,"threads_started":1}
I20260807 15:11:25.040648 31682 tablet_service.cc:1467] Processing CreateTablet for tablet 31249001c12b4c5fa1e54a48e6e489e9 (DEFAULT_TABLE table=test-table [id=70a8561f90114961af20f1749ba45849]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:11:25.040940 32005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:25.035223 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34582 (local address 127.29.194.1:45001)
0807 15:11:25.035436 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:25.035440 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:25.035648 (+ 208us) server_negotiation.cc:407] Connection header received
0807 15:11:25.035750 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:25.035753 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:25.035770 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:25.035861 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:11:25.036004 (+ 143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:25.037005 (+ 1001us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:25.039665 (+ 2660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:25.040393 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:25.040480 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:25.040542 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0807 15:11:25.040793 (+ 251us) server_negotiation.cc:299] Negotiation successful
0807 15:11:25.040845 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":184,"thread_start_us":104,"threads_started":1}
I20260807 15:11:25.040946 32001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:25.034991 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.1:45001 (local address 127.0.0.1:34582)
0807 15:11:25.035497 (+ 506us) negotiation.cc:107] Waiting for socket to connect
0807 15:11:25.035506 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0807 15:11:25.035659 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:11:25.035875 (+ 216us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:11:25.035876 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:11:25.035878 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:11:25.035933 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:25.035934 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:25.038484 (+ 2550us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:25.038485 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:25.039571 (+ 1086us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:25.039574 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:25.040623 (+ 1049us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:25.040624 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:25.040694 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:25.040700 (+ 6us) client_negotiation.cc:769] Sending connection context
0807 15:11:25.040766 (+ 66us) client_negotiation.cc:240] Negotiation successful
0807 15:11:25.040809 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":490,"thread_start_us":81,"threads_started":1}
I20260807 15:11:25.041132 31682 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 31249001c12b4c5fa1e54a48e6e489e9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:11:25.041049 31880 tablet_service.cc:1467] Processing CreateTablet for tablet 31249001c12b4c5fa1e54a48e6e489e9 (DEFAULT_TABLE table=test-table [id=70a8561f90114961af20f1749ba45849]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:11:25.041379 31880 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 31249001c12b4c5fa1e54a48e6e489e9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:11:25.042332 31485 tablet_service.cc:1467] Processing CreateTablet for tablet 31249001c12b4c5fa1e54a48e6e489e9 (DEFAULT_TABLE table=test-table [id=70a8561f90114961af20f1749ba45849]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:11:25.042708 31485 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 31249001c12b4c5fa1e54a48e6e489e9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:11:25.044812 32006 tablet_bootstrap.cc:492] T 31249001c12b4c5fa1e54a48e6e489e9 P 9cde9d7f2650466499b01c1b3f53bf69: Bootstrap starting.
I20260807 15:11:25.045181 32007 tablet_bootstrap.cc:492] T 31249001c12b4c5fa1e54a48e6e489e9 P d20885f827b64f90a4004ff81c309ae0: Bootstrap starting.
I20260807 15:11:25.045717 32006 tablet_bootstrap.cc:654] T 31249001c12b4c5fa1e54a48e6e489e9 P 9cde9d7f2650466499b01c1b3f53bf69: Neither blocks nor log segments found. Creating new log.
I20260807 15:11:25.045815 32007 tablet_bootstrap.cc:654] T 31249001c12b4c5fa1e54a48e6e489e9 P d20885f827b64f90a4004ff81c309ae0: Neither blocks nor log segments found. Creating new log.
I20260807 15:11:25.046248 32008 tablet_bootstrap.cc:492] T 31249001c12b4c5fa1e54a48e6e489e9 P 8dff9d4f4e624e2186a57a2085f236d3: Bootstrap starting.
I20260807 15:11:25.046402 32006 log.cc:824] T 31249001c12b4c5fa1e54a48e6e489e9 P 9cde9d7f2650466499b01c1b3f53bf69: Log is configured to *not* fsync() on all Append() calls
I20260807 15:11:25.047016 32008 tablet_bootstrap.cc:654] T 31249001c12b4c5fa1e54a48e6e489e9 P 8dff9d4f4e624e2186a57a2085f236d3: Neither blocks nor log segments found. Creating new log.
I20260807 15:11:25.047163 32006 tablet_bootstrap.cc:492] T 31249001c12b4c5fa1e54a48e6e489e9 P 9cde9d7f2650466499b01c1b3f53bf69: No bootstrap required, opened a new log
I20260807 15:11:25.047220 32006 ts_tablet_manager.cc:1397] T 31249001c12b4c5fa1e54a48e6e489e9 P 9cde9d7f2650466499b01c1b3f53bf69: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260807 15:11:25.047518 32008 log.cc:824] T 31249001c12b4c5fa1e54a48e6e489e9 P 8dff9d4f4e624e2186a57a2085f236d3: Log is configured to *not* fsync() on all Append() calls
I20260807 15:11:25.047719 32007 log.cc:824] T 31249001c12b4c5fa1e54a48e6e489e9 P d20885f827b64f90a4004ff81c309ae0: Log is configured to *not* fsync() on all Append() calls
I20260807 15:11:25.048269 32008 tablet_bootstrap.cc:492] T 31249001c12b4c5fa1e54a48e6e489e9 P 8dff9d4f4e624e2186a57a2085f236d3: No bootstrap required, opened a new log
I20260807 15:11:25.048336 32008 ts_tablet_manager.cc:1397] T 31249001c12b4c5fa1e54a48e6e489e9 P 8dff9d4f4e624e2186a57a2085f236d3: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 15:11:25.049000 32007 tablet_bootstrap.cc:492] T 31249001c12b4c5fa1e54a48e6e489e9 P d20885f827b64f90a4004ff81c309ae0: No bootstrap required, opened a new log
I20260807 15:11:25.049078 32007 ts_tablet_manager.cc:1397] T 31249001c12b4c5fa1e54a48e6e489e9 P d20885f827b64f90a4004ff81c309ae0: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.003s
I20260807 15:11:25.049041 32006 raft_consensus.cc:348] T 31249001c12b4c5fa1e54a48e6e489e9 P 9cde9d7f2650466499b01c1b3f53bf69 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cde9d7f2650466499b01c1b3f53bf69" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 39825 } } peers { permanent_uuid: "8dff9d4f4e624e2186a57a2085f236d3" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 45001 } } peers { permanent_uuid: "d20885f827b64f90a4004ff81c309ae0" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 34407 } }
I20260807 15:11:25.049226 32006 raft_consensus.cc:374] T 31249001c12b4c5fa1e54a48e6e489e9 P 9cde9d7f2650466499b01c1b3f53bf69 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:11:25.049269 32006 raft_consensus.cc:729] T 31249001c12b4c5fa1e54a48e6e489e9 P 9cde9d7f2650466499b01c1b3f53bf69 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9cde9d7f2650466499b01c1b3f53bf69, State: Initialized, Role: FOLLOWER
I20260807 15:11:25.049408 32006 consensus_queue.cc:260] T 31249001c12b4c5fa1e54a48e6e489e9 P 9cde9d7f2650466499b01c1b3f53bf69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cde9d7f2650466499b01c1b3f53bf69" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 39825 } } peers { permanent_uuid: "8dff9d4f4e624e2186a57a2085f236d3" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 45001 } } peers { permanent_uuid: "d20885f827b64f90a4004ff81c309ae0" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 34407 } }
I20260807 15:11:25.049728 32006 ts_tablet_manager.cc:1428] T 31249001c12b4c5fa1e54a48e6e489e9 P 9cde9d7f2650466499b01c1b3f53bf69: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260807 15:11:25.049875 31981 heartbeater.cc:503] Master 127.29.194.62:42305 was elected leader, sending a full tablet report...
I20260807 15:11:25.050172 32008 raft_consensus.cc:348] T 31249001c12b4c5fa1e54a48e6e489e9 P 8dff9d4f4e624e2186a57a2085f236d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cde9d7f2650466499b01c1b3f53bf69" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 39825 } } peers { permanent_uuid: "8dff9d4f4e624e2186a57a2085f236d3" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 45001 } } peers { permanent_uuid: "d20885f827b64f90a4004ff81c309ae0" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 34407 } }
I20260807 15:11:25.050328 32008 raft_consensus.cc:374] T 31249001c12b4c5fa1e54a48e6e489e9 P 8dff9d4f4e624e2186a57a2085f236d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:11:25.050366 32008 raft_consensus.cc:729] T 31249001c12b4c5fa1e54a48e6e489e9 P 8dff9d4f4e624e2186a57a2085f236d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8dff9d4f4e624e2186a57a2085f236d3, State: Initialized, Role: FOLLOWER
I20260807 15:11:25.050474 32008 consensus_queue.cc:260] T 31249001c12b4c5fa1e54a48e6e489e9 P 8dff9d4f4e624e2186a57a2085f236d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cde9d7f2650466499b01c1b3f53bf69" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 39825 } } peers { permanent_uuid: "8dff9d4f4e624e2186a57a2085f236d3" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 45001 } } peers { permanent_uuid: "d20885f827b64f90a4004ff81c309ae0" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 34407 } }
I20260807 15:11:25.050784 32008 ts_tablet_manager.cc:1428] T 31249001c12b4c5fa1e54a48e6e489e9 P 8dff9d4f4e624e2186a57a2085f236d3: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260807 15:11:25.050796 31578 heartbeater.cc:503] Master 127.29.194.62:42305 was elected leader, sending a full tablet report...
I20260807 15:11:25.051184 32007 raft_consensus.cc:348] T 31249001c12b4c5fa1e54a48e6e489e9 P d20885f827b64f90a4004ff81c309ae0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cde9d7f2650466499b01c1b3f53bf69" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 39825 } } peers { permanent_uuid: "8dff9d4f4e624e2186a57a2085f236d3" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 45001 } } peers { permanent_uuid: "d20885f827b64f90a4004ff81c309ae0" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 34407 } }
I20260807 15:11:25.051370 32007 raft_consensus.cc:374] T 31249001c12b4c5fa1e54a48e6e489e9 P d20885f827b64f90a4004ff81c309ae0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:11:25.051417 32007 raft_consensus.cc:729] T 31249001c12b4c5fa1e54a48e6e489e9 P d20885f827b64f90a4004ff81c309ae0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d20885f827b64f90a4004ff81c309ae0, State: Initialized, Role: FOLLOWER
I20260807 15:11:25.051532 32007 consensus_queue.cc:260] T 31249001c12b4c5fa1e54a48e6e489e9 P d20885f827b64f90a4004ff81c309ae0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cde9d7f2650466499b01c1b3f53bf69" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 39825 } } peers { permanent_uuid: "8dff9d4f4e624e2186a57a2085f236d3" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 45001 } } peers { permanent_uuid: "d20885f827b64f90a4004ff81c309ae0" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 34407 } }
I20260807 15:11:25.051822 32007 ts_tablet_manager.cc:1428] T 31249001c12b4c5fa1e54a48e6e489e9 P d20885f827b64f90a4004ff81c309ae0: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260807 15:11:25.051945 31784 heartbeater.cc:503] Master 127.29.194.62:42305 was elected leader, sending a full tablet report...
W20260807 15:11:25.074517 31786 tablet.cc:2369] T 31249001c12b4c5fa1e54a48e6e489e9 P d20885f827b64f90a4004ff81c309ae0: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 15:11:25.140394 31579 tablet.cc:2369] T 31249001c12b4c5fa1e54a48e6e489e9 P 8dff9d4f4e624e2186a57a2085f236d3: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 15:11:25.249114 31982 tablet.cc:2369] T 31249001c12b4c5fa1e54a48e6e489e9 P 9cde9d7f2650466499b01c1b3f53bf69: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 15:11:25.296931 32013 raft_consensus.cc:482] T 31249001c12b4c5fa1e54a48e6e489e9 P 8dff9d4f4e624e2186a57a2085f236d3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 15:11:25.297075 32013 raft_consensus.cc:504] T 31249001c12b4c5fa1e54a48e6e489e9 P 8dff9d4f4e624e2186a57a2085f236d3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cde9d7f2650466499b01c1b3f53bf69" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 39825 } } peers { permanent_uuid: "8dff9d4f4e624e2186a57a2085f236d3" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 45001 } } peers { permanent_uuid: "d20885f827b64f90a4004ff81c309ae0" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 34407 } }
I20260807 15:11:25.297420 32013 leader_election.cc:288] T 31249001c12b4c5fa1e54a48e6e489e9 P 8dff9d4f4e624e2186a57a2085f236d3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9cde9d7f2650466499b01c1b3f53bf69 (127.29.194.3:39825), d20885f827b64f90a4004ff81c309ae0 (127.29.194.2:34407)
I20260807 15:11:25.300498 31998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:25.297838 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.194.1:37035 (local address 127.29.194.3:39825)
0807 15:11:25.297919 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:25.297921 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:25.297972 (+ 51us) server_negotiation.cc:407] Connection header received
0807 15:11:25.298039 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:25.298040 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:25.298051 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:25.298101 (+ 50us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:11:25.298256 (+ 155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:25.298738 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:25.299786 (+ 1048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:25.300175 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:25.300238 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:25.300247 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0807 15:11:25.300381 (+ 134us) server_negotiation.cc:299] Negotiation successful
0807 15:11:25.300419 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260807 15:11:25.300498 32002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:25.297812 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.194.1:50945 (local address 127.29.194.2:34407)
0807 15:11:25.297886 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:25.297888 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:25.297996 (+ 108us) server_negotiation.cc:407] Connection header received
0807 15:11:25.298065 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:25.298066 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:25.298078 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:25.298124 (+ 46us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:11:25.298297 (+ 173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:25.298758 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:25.299793 (+ 1035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:25.300168 (+ 375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:25.300232 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:25.300242 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0807 15:11:25.300386 (+ 144us) server_negotiation.cc:299] Negotiation successful
0807 15:11:25.300427 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260807 15:11:25.300541 32016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:25.297725 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.3:39825 (local address 127.29.194.1:37035)
0807 15:11:25.297895 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0807 15:11:25.297908 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 15:11:25.297983 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:11:25.298112 (+ 129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:11:25.298113 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:11:25.298114 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:11:25.298194 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:25.298196 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:25.298819 (+ 623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:25.298821 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:25.299695 (+ 874us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:25.299699 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:25.300254 (+ 555us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:25.300255 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:25.300322 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:25.300325 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 15:11:25.300377 (+ 52us) client_negotiation.cc:240] Negotiation successful
0807 15:11:25.300419 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"mutex_wait_us":38,"thread_start_us":76,"threads_started":1}
I20260807 15:11:25.300554 32015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:25.297741 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.2:34407 (local address 127.29.194.1:50945)
0807 15:11:25.297919 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0807 15:11:25.297929 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 15:11:25.297996 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:11:25.298137 (+ 141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:11:25.298138 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:11:25.298140 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:11:25.298228 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:25.298230 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:25.298840 (+ 610us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:25.298841 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:25.299698 (+ 857us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:25.299700 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:25.300248 (+ 548us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:25.300248 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:25.300320 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:25.300324 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 15:11:25.300370 (+ 46us) client_negotiation.cc:240] Negotiation successful
0807 15:11:25.300418 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":165,"mutex_wait_us":17,"thread_start_us":70,"threads_started":1}
I20260807 15:11:25.300956 31912 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "31249001c12b4c5fa1e54a48e6e489e9" candidate_uuid: "8dff9d4f4e624e2186a57a2085f236d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9cde9d7f2650466499b01c1b3f53bf69" is_pre_election: true
I20260807 15:11:25.301147 31912 raft_consensus.cc:2454] T 31249001c12b4c5fa1e54a48e6e489e9 P 9cde9d7f2650466499b01c1b3f53bf69 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8dff9d4f4e624e2186a57a2085f236d3 in term 0.
I20260807 15:11:25.301267 31711 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "31249001c12b4c5fa1e54a48e6e489e9" candidate_uuid: "8dff9d4f4e624e2186a57a2085f236d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d20885f827b64f90a4004ff81c309ae0" is_pre_election: true
I20260807 15:11:25.301409 31414 leader_election.cc:302] T 31249001c12b4c5fa1e54a48e6e489e9 P 8dff9d4f4e624e2186a57a2085f236d3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8dff9d4f4e624e2186a57a2085f236d3, 9cde9d7f2650466499b01c1b3f53bf69; no voters:
I20260807 15:11:25.301435 31711 raft_consensus.cc:2454] T 31249001c12b4c5fa1e54a48e6e489e9 P d20885f827b64f90a4004ff81c309ae0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8dff9d4f4e624e2186a57a2085f236d3 in term 0.
I20260807 15:11:25.301586 32013 raft_consensus.cc:2781] T 31249001c12b4c5fa1e54a48e6e489e9 P 8dff9d4f4e624e2186a57a2085f236d3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 15:11:25.301638 32013 raft_consensus.cc:482] T 31249001c12b4c5fa1e54a48e6e489e9 P 8dff9d4f4e624e2186a57a2085f236d3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 15:11:25.301661 32013 raft_consensus.cc:3037] T 31249001c12b4c5fa1e54a48e6e489e9 P 8dff9d4f4e624e2186a57a2085f236d3 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:11:25.302443 32013 raft_consensus.cc:504] T 31249001c12b4c5fa1e54a48e6e489e9 P 8dff9d4f4e624e2186a57a2085f236d3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cde9d7f2650466499b01c1b3f53bf69" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 39825 } } peers { permanent_uuid: "8dff9d4f4e624e2186a57a2085f236d3" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 45001 } } peers { permanent_uuid: "d20885f827b64f90a4004ff81c309ae0" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 34407 } }
I20260807 15:11:25.302591 32013 leader_election.cc:288] T 31249001c12b4c5fa1e54a48e6e489e9 P 8dff9d4f4e624e2186a57a2085f236d3 [CANDIDATE]: Term 1 election: Requested vote from peers 9cde9d7f2650466499b01c1b3f53bf69 (127.29.194.3:39825), d20885f827b64f90a4004ff81c309ae0 (127.29.194.2:34407)
I20260807 15:11:25.302754 31711 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "31249001c12b4c5fa1e54a48e6e489e9" candidate_uuid: "8dff9d4f4e624e2186a57a2085f236d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d20885f827b64f90a4004ff81c309ae0"
I20260807 15:11:25.302836 31711 raft_consensus.cc:3037] T 31249001c12b4c5fa1e54a48e6e489e9 P d20885f827b64f90a4004ff81c309ae0 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:11:25.302964 31912 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "31249001c12b4c5fa1e54a48e6e489e9" candidate_uuid: "8dff9d4f4e624e2186a57a2085f236d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9cde9d7f2650466499b01c1b3f53bf69"
I20260807 15:11:25.303097 31912 raft_consensus.cc:3037] T 31249001c12b4c5fa1e54a48e6e489e9 P 9cde9d7f2650466499b01c1b3f53bf69 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:11:25.303769 31711 raft_consensus.cc:2454] T 31249001c12b4c5fa1e54a48e6e489e9 P d20885f827b64f90a4004ff81c309ae0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8dff9d4f4e624e2186a57a2085f236d3 in term 1.
I20260807 15:11:25.304040 31411 leader_election.cc:302] T 31249001c12b4c5fa1e54a48e6e489e9 P 8dff9d4f4e624e2186a57a2085f236d3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8dff9d4f4e624e2186a57a2085f236d3, d20885f827b64f90a4004ff81c309ae0; no voters:
I20260807 15:11:25.304072 31912 raft_consensus.cc:2454] T 31249001c12b4c5fa1e54a48e6e489e9 P 9cde9d7f2650466499b01c1b3f53bf69 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8dff9d4f4e624e2186a57a2085f236d3 in term 1.
I20260807 15:11:25.304163 32013 raft_consensus.cc:2781] T 31249001c12b4c5fa1e54a48e6e489e9 P 8dff9d4f4e624e2186a57a2085f236d3 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:11:25.304349 32013 raft_consensus.cc:686] T 31249001c12b4c5fa1e54a48e6e489e9 P 8dff9d4f4e624e2186a57a2085f236d3 [term 1 LEADER]: Becoming Leader. State: Replica: 8dff9d4f4e624e2186a57a2085f236d3, State: Running, Role: LEADER
I20260807 15:11:25.304467 32013 consensus_queue.cc:237] T 31249001c12b4c5fa1e54a48e6e489e9 P 8dff9d4f4e624e2186a57a2085f236d3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cde9d7f2650466499b01c1b3f53bf69" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 39825 } } peers { permanent_uuid: "8dff9d4f4e624e2186a57a2085f236d3" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 45001 } } peers { permanent_uuid: "d20885f827b64f90a4004ff81c309ae0" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 34407 } }
I20260807 15:11:25.305604 31309 catalog_manager.cc:5495] T 31249001c12b4c5fa1e54a48e6e489e9 P 8dff9d4f4e624e2186a57a2085f236d3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8dff9d4f4e624e2186a57a2085f236d3 (127.29.194.1). New cstate: current_term: 1 leader_uuid: "8dff9d4f4e624e2186a57a2085f236d3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cde9d7f2650466499b01c1b3f53bf69" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 39825 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8dff9d4f4e624e2186a57a2085f236d3" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 45001 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d20885f827b64f90a4004ff81c309ae0" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 34407 } health_report { overall_health: UNKNOWN } } }
Aug 07 15:11:25 dist-test-slave-42z9 krb5kdc[31245](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115484, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.194.1@KRBTEST.COM
I20260807 15:11:25.335448 32005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:25.326940 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34598 (local address 127.29.194.1:45001)
0807 15:11:25.327010 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:25.327012 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:25.327022 (+ 10us) server_negotiation.cc:407] Connection header received
0807 15:11:25.327099 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:25.327100 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:25.327112 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:25.327162 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:11:25.327784 (+ 622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:25.328278 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:25.329123 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:25.329376 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:25.329553 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:25.331951 (+ 2398us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:11:25.331969 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:11:25.331970 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:11:25.331981 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:11:25.333783 (+ 1802us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:25.334232 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:25.334237 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:25.334238 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:25.334380 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:25.334613 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:25.334615 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:25.334615 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:25.334809 (+ 194us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:11:25.334951 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0807 15:11:25.335190 (+ 239us) server_negotiation.cc:299] Negotiation successful
0807 15:11:25.335371 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
Aug 07 15:11:25 dist-test-slave-42z9 krb5kdc[31245](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115485, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 07 15:11:25 dist-test-slave-42z9 krb5kdc[31245](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115485, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.29.194.1@KRBTEST.COM
I20260807 15:11:25.364235 32005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:25.356837 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34614 (local address 127.29.194.1:45001)
0807 15:11:25.356914 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:25.356916 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:25.356961 (+ 45us) server_negotiation.cc:407] Connection header received
0807 15:11:25.357035 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:25.357036 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:25.357047 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:25.357096 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:11:25.357743 (+ 647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:25.358227 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:25.358936 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:25.359127 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:25.359184 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:25.361407 (+ 2223us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:11:25.361408 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:11:25.361409 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:11:25.361417 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:11:25.362702 (+ 1285us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:25.363015 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:25.363016 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:25.363017 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:25.363069 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:25.363451 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:25.363453 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:25.363454 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:25.363639 (+ 185us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:11:25.363794 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0807 15:11:25.363992 (+ 198us) server_negotiation.cc:299] Negotiation successful
0807 15:11:25.364112 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
Aug 07 15:11:25 dist-test-slave-42z9 krb5kdc[31245](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115485, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 07 15:11:25 dist-test-slave-42z9 krb5kdc[31245](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115485, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.194.1@KRBTEST.COM
I20260807 15:11:25.390543 32005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:25.382925 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34626 (local address 127.29.194.1:45001)
0807 15:11:25.383020 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:25.383023 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:25.383201 (+ 178us) server_negotiation.cc:407] Connection header received
0807 15:11:25.383274 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:25.383276 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:25.383291 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:25.383354 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:11:25.384009 (+ 655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:25.384509 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:25.385278 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:25.385521 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:25.385580 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:25.388320 (+ 2740us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:11:25.388322 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:11:25.388323 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:11:25.388333 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:11:25.389583 (+ 1250us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:25.389940 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:25.389940 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:25.389941 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:25.389998 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:25.390141 (+ 143us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:25.390141 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:25.390142 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:25.390240 (+ 98us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:11:25.390342 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0807 15:11:25.390399 (+ 57us) server_negotiation.cc:299] Negotiation successful
0807 15:11:25.390470 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
W20260807 15:11:25.391597 31455 tablet_service.cc:2908] Rejecting scan request for tablet 31249001c12b4c5fa1e54a48e6e489e9: Uninitialized: clean time has not yet been initialized
I20260807 15:11:25.393090 30472 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7fZSIL/build/release/bin/kudu with pid 31388
I20260807 15:11:25.405694 30472 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7fZSIL/build/release/bin/kudu with pid 31586
I20260807 15:11:25.421300 30472 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7fZSIL/build/release/bin/kudu with pid 31790
I20260807 15:11:25.427098 30472 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7fZSIL/build/release/bin/kudu with pid 31260
2026-08-07T15:11:25Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (2424 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115480851507-30472-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 15:11:25 dist-test-slave-42z9 krb5kdc[32035](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 15:11:25 dist-test-slave-42z9 krb5kdc[32035](info): set up 2 sockets
Aug 07 15:11:25 dist-test-slave-42z9 krb5kdc[32035](info): commencing operation
krb5kdc: starting...
W20260807 15:11:26.598342 30472 mini_kdc.cc:121] Time spent starting KDC: real 1.142s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115480851507-30472-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115480851507-30472-0/krb5kdc/test-user.keytab.
Aug 07 15:11:26 dist-test-slave-42z9 krb5kdc[32035](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115486, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T15:11:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T15:11:26Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.194.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.194.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.194.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.62.keytab.
Entry for principal kudu/127.29.194.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.62.keytab.
I20260807 15:11:26.738884 30472 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7fZSIL/build/release/bin/kudu
/tmp/dist-test-task7fZSIL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115480851507-30472-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115480851507-30472-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115480851507-30472-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115480851507-30472-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.194.62:45747
--webserver_interface=127.29.194.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.62.keytab
--principal=kudu/127.29.194.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.194.20:36373
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.194.62:45747
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115480851507-30472-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115480851507-30472-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115480851507-30472-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:11:26.831255 32058 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:11:26.831509 32058 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:11:26.831549 32058 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:11:26.832870 32058 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:11:26.832931 32058 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260807 15:11:26.832948 32058 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:11:26.832965 32058 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:11:26.832981 32058 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:11:26.832998 32058 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:11:26.834805 32058 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.194.20:36373
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115480851507-30472-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115480851507-30472-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.194.62:45747
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.62.keytab
--principal=kudu/127.29.194.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.194.62:45747
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115480851507-30472-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.194.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32058
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115480851507-30472-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:11:26.835119 32058 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:11:26.835356 32058 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:11:26.840792 32066 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:11:26.841056 32058 server_base.cc:1034] running on GCE node
W20260807 15:11:26.840930 32065 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:11:26.840950 32068 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:11:26.841598 32058 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:11:26.841960 32058 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:11:26.843137 32058 hybrid_clock.cc:648] HybridClock initialized: now 1786115486843120 us; error 34 us; skew 500 ppm
Aug 07 15:11:26 dist-test-slave-42z9 krb5kdc[32035](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115486, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.194.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:11:26.850831 32058 init.cc:377] Logged in from keytab as kudu/127.29.194.62@KRBTEST.COM (short username kudu)
I20260807 15:11:26.851352 32058 webserver.cc:460] Webserver started at http://127.29.194.62:41271/ using document root <none> and password file <none>
I20260807 15:11:26.851586 32058 fs_manager.cc:359] Metadata directory not provided
I20260807 15:11:26.851637 32058 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:11:26.851776 32058 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:11:26.852883 32058 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115480851507-30472-0/minicluster-data/master-0/data/instance:
uuid: "b676891f78094012accc7df16ace058c"
format_stamp: "Formatted at 2026-08-07 15:11:26 on dist-test-slave-42z9"
server_key: "0f304f3c4a77bf5107c235313a7fb743"
server_key_iv: "ca1670c641af7cf117cc38b3c95d600b"
server_key_version: "encryptionkey@0"
I20260807 15:11:26.853256 32058 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115480851507-30472-0/minicluster-data/master-0/wal/instance:
uuid: "b676891f78094012accc7df16ace058c"
format_stamp: "Formatted at 2026-08-07 15:11:26 on dist-test-slave-42z9"
server_key: "0f304f3c4a77bf5107c235313a7fb743"
server_key_iv: "ca1670c641af7cf117cc38b3c95d600b"
server_key_version: "encryptionkey@0"
I20260807 15:11:26.854764 32058 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260807 15:11:26.855827 32078 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:11:26.856145 32058 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:11:26.856249 32058 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115480851507-30472-0/minicluster-data/master-0/data,/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115480851507-30472-0/minicluster-data/master-0/wal
uuid: "b676891f78094012accc7df16ace058c"
format_stamp: "Formatted at 2026-08-07 15:11:26 on dist-test-slave-42z9"
server_key: "0f304f3c4a77bf5107c235313a7fb743"
server_key_iv: "ca1670c641af7cf117cc38b3c95d600b"
server_key_version: "encryptionkey@0"
I20260807 15:11:26.856329 32058 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115480851507-30472-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115480851507-30472-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115480851507-30472-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:11:26.875181 32058 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:11:26.877554 32058 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:11:26.877728 32058 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:11:26.883201 32058 rpc_server.cc:307] RPC server started. Bound to: 127.29.194.62:45747
I20260807 15:11:26.883263 32164 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.194.62:45747 every 8 connection(s)
I20260807 15:11:26.883659 32058 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115480851507-30472-0/minicluster-data/master-0/data/info.pb
I20260807 15:11:26.884446 32165 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:11:26.887096 32165 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b676891f78094012accc7df16ace058c: Bootstrap starting.
I20260807 15:11:26.887252 30472 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7fZSIL/build/release/bin/kudu as pid 32058
I20260807 15:11:26.887352 30472 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115480851507-30472-0/minicluster-data/master-0/wal/instance
I20260807 15:11:26.887521 30472 external_mini_cluster.cc:1428] Setting key 251a6516605d957b2de81f1b10559d69
I20260807 15:11:26.887897 32165 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b676891f78094012accc7df16ace058c: Neither blocks nor log segments found. Creating new log.
I20260807 15:11:26.888497 32165 log.cc:824] T 00000000000000000000000000000000 P b676891f78094012accc7df16ace058c: Log is configured to *not* fsync() on all Append() calls
I20260807 15:11:26.889384 32165 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b676891f78094012accc7df16ace058c: No bootstrap required, opened a new log
I20260807 15:11:26.891820 32165 raft_consensus.cc:348] T 00000000000000000000000000000000 P b676891f78094012accc7df16ace058c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b676891f78094012accc7df16ace058c" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 45747 } }
I20260807 15:11:26.892001 32165 raft_consensus.cc:374] T 00000000000000000000000000000000 P b676891f78094012accc7df16ace058c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:11:26.892024 32165 raft_consensus.cc:729] T 00000000000000000000000000000000 P b676891f78094012accc7df16ace058c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b676891f78094012accc7df16ace058c, State: Initialized, Role: FOLLOWER
I20260807 15:11:26.892170 32165 consensus_queue.cc:260] T 00000000000000000000000000000000 P b676891f78094012accc7df16ace058c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b676891f78094012accc7df16ace058c" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 45747 } }
I20260807 15:11:26.892259 32165 raft_consensus.cc:388] T 00000000000000000000000000000000 P b676891f78094012accc7df16ace058c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:11:26.892302 32165 raft_consensus.cc:482] T 00000000000000000000000000000000 P b676891f78094012accc7df16ace058c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:11:26.892373 32165 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b676891f78094012accc7df16ace058c [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:11:26.893020 32165 raft_consensus.cc:504] T 00000000000000000000000000000000 P b676891f78094012accc7df16ace058c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b676891f78094012accc7df16ace058c" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 45747 } }
I20260807 15:11:26.893147 32165 leader_election.cc:302] T 00000000000000000000000000000000 P b676891f78094012accc7df16ace058c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b676891f78094012accc7df16ace058c; no voters:
I20260807 15:11:26.893364 32165 leader_election.cc:288] T 00000000000000000000000000000000 P b676891f78094012accc7df16ace058c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 15:11:26.893410 32174 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b676891f78094012accc7df16ace058c [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:11:26.893577 32174 raft_consensus.cc:686] T 00000000000000000000000000000000 P b676891f78094012accc7df16ace058c [term 1 LEADER]: Becoming Leader. State: Replica: b676891f78094012accc7df16ace058c, State: Running, Role: LEADER
I20260807 15:11:26.893697 32165 sys_catalog.cc:564] T 00000000000000000000000000000000 P b676891f78094012accc7df16ace058c [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:11:26.893676 32174 consensus_queue.cc:237] T 00000000000000000000000000000000 P b676891f78094012accc7df16ace058c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b676891f78094012accc7df16ace058c" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 45747 } }
I20260807 15:11:26.895462 32175 sys_catalog.cc:455] T 00000000000000000000000000000000 P b676891f78094012accc7df16ace058c [sys.catalog]: SysCatalogTable state changed. Reason: New leader b676891f78094012accc7df16ace058c. Latest consensus state: current_term: 1 leader_uuid: "b676891f78094012accc7df16ace058c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b676891f78094012accc7df16ace058c" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 45747 } } }
I20260807 15:11:26.895578 32175 sys_catalog.cc:458] T 00000000000000000000000000000000 P b676891f78094012accc7df16ace058c [sys.catalog]: This master's current role is: LEADER
I20260807 15:11:26.895807 32176 sys_catalog.cc:455] T 00000000000000000000000000000000 P b676891f78094012accc7df16ace058c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b676891f78094012accc7df16ace058c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b676891f78094012accc7df16ace058c" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 45747 } } }
I20260807 15:11:26.895915 32176 sys_catalog.cc:458] T 00000000000000000000000000000000 P b676891f78094012accc7df16ace058c [sys.catalog]: This master's current role is: LEADER
Aug 07 15:11:26 dist-test-slave-42z9 krb5kdc[32035](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115486, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.194.62@KRBTEST.COM
I20260807 15:11:26.896090 32181 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:11:26.896764 32181 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:11:26.899173 32181 catalog_manager.cc:1347] Generated new cluster ID: 3e2cdb43315e46769e004e45bc5fd813
I20260807 15:11:26.899236 32181 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:11:26.901423 32170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:26.888088 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43836 (local address 127.29.194.62:45747)
0807 15:11:26.888286 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:26.888295 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:26.888336 (+ 41us) server_negotiation.cc:407] Connection header received
0807 15:11:26.888967 (+ 631us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:26.888975 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:26.889040 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:26.889201 (+ 161us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:11:26.889869 (+ 668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:26.891256 (+ 1387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:26.892417 (+ 1161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:26.893079 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:26.893246 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:26.896701 (+ 3455us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:11:26.896706 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:11:26.896707 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:11:26.896722 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:11:26.898918 (+ 2196us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:26.899466 (+ 548us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:26.899469 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:26.899470 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:26.899583 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:26.899762 (+ 179us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:26.899762 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:26.899763 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:26.900066 (+ 303us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:11:26.900213 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0807 15:11:26.900725 (+ 512us) server_negotiation.cc:299] Negotiation successful
0807 15:11:26.900880 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":138,"thread_start_us":78,"threads_started":1}
I20260807 15:11:26.928674 32181 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:11:26.929268 32181 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:11:26.932579 32181 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b676891f78094012accc7df16ace058c: Generated new TSK 0
I20260807 15:11:26.932919 32181 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.29.194.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.194.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.194.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.1.keytab.
Entry for principal kudu/127.29.194.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.1.keytab.
I20260807 15:11:26.968891 30472 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7fZSIL/build/release/bin/kudu
/tmp/dist-test-task7fZSIL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115480851507-30472-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115480851507-30472-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115480851507-30472-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115480851507-30472-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.194.1:0
--local_ip_for_outbound_sockets=127.29.194.1
--webserver_interface=127.29.194.1
--webserver_port=0
--tserver_master_addrs=127.29.194.62:45747
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.1.keytab
--principal=kudu/127.29.194.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.194.20:36373
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115480851507-30472-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115480851507-30472-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115480851507-30472-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:11:27.064363 32201 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:11:27.064606 32201 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:11:27.064644 32201 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:11:27.066043 32201 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:11:27.066102 32201 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:11:27.066133 32201 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 15:11:27.066159 32201 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.194.1
I20260807 15:11:27.067833 32201 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.194.20:36373
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115480851507-30472-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115480851507-30472-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.1.keytab
--principal=kudu/127.29.194.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.194.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115480851507-30472-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.194.1
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.29.194.62:45747
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32201
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.194.1
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115480851507-30472-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:11:27.068120 32201 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:11:27.068356 32201 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:11:27.073911 32209 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:11:27.074043 32201 server_base.cc:1034] running on GCE node
W20260807 15:11:27.073922 32210 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:11:27.073922 32217 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:11:27.074433 32201 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:11:27.074697 32201 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:11:27.075879 32201 hybrid_clock.cc:648] HybridClock initialized: now 1786115487075830 us; error 63 us; skew 500 ppm
Aug 07 15:11:27 dist-test-slave-42z9 krb5kdc[32035](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115487, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.194.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:11:27.079281 32201 init.cc:377] Logged in from keytab as kudu/127.29.194.1@KRBTEST.COM (short username kudu)
I20260807 15:11:27.079799 32201 webserver.cc:460] Webserver started at http://127.29.194.1:42945/ using document root <none> and password file <none>
I20260807 15:11:27.080049 32201 fs_manager.cc:359] Metadata directory not provided
I20260807 15:11:27.080102 32201 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:11:27.080231 32201 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:11:27.081363 32201 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115480851507-30472-0/minicluster-data/ts-0/data/instance:
uuid: "62c4c4f74c3744ddbc461ef9b6acfad4"
format_stamp: "Formatted at 2026-08-07 15:11:27 on dist-test-slave-42z9"
server_key: "1f26e2cb64725df82e6811e019a03a5c"
server_key_iv: "3b634b170cdb1b0cb33344f775cd7b38"
server_key_version: "encryptionkey@0"
I20260807 15:11:27.081784 32201 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115480851507-30472-0/minicluster-data/ts-0/wal/instance:
uuid: "62c4c4f74c3744ddbc461ef9b6acfad4"
format_stamp: "Formatted at 2026-08-07 15:11:27 on dist-test-slave-42z9"
server_key: "1f26e2cb64725df82e6811e019a03a5c"
server_key_iv: "3b634b170cdb1b0cb33344f775cd7b38"
server_key_version: "encryptionkey@0"
I20260807 15:11:27.083199 32201 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:11:27.084074 32228 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:11:27.084282 32201 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:11:27.084425 32201 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115480851507-30472-0/minicluster-data/ts-0/data,/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115480851507-30472-0/minicluster-data/ts-0/wal
uuid: "62c4c4f74c3744ddbc461ef9b6acfad4"
format_stamp: "Formatted at 2026-08-07 15:11:27 on dist-test-slave-42z9"
server_key: "1f26e2cb64725df82e6811e019a03a5c"
server_key_iv: "3b634b170cdb1b0cb33344f775cd7b38"
server_key_version: "encryptionkey@0"
I20260807 15:11:27.084502 32201 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115480851507-30472-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115480851507-30472-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115480851507-30472-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:11:27.093246 32201 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:11:27.096647 32201 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:11:27.096809 32201 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:11:27.097383 32201 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:11:27.097424 32201 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:11:27.097486 32201 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:11:27.097507 32201 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:11:27.104897 32201 rpc_server.cc:307] RPC server started. Bound to: 127.29.194.1:35187
I20260807 15:11:27.105068 32393 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.194.1:35187 every 8 connection(s)
I20260807 15:11:27.105352 32201 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115480851507-30472-0/minicluster-data/ts-0/data/info.pb
Aug 07 15:11:27 dist-test-slave-42z9 krb5kdc[32035](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115487, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.194.1@KRBTEST.COM for kudu/127.29.194.62@KRBTEST.COM
I20260807 15:11:27.106144 30472 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7fZSIL/build/release/bin/kudu as pid 32201
I20260807 15:11:27.106227 30472 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115480851507-30472-0/minicluster-data/ts-0/wal/instance
I20260807 15:11:27.106390 30472 external_mini_cluster.cc:1428] Setting key 350cc8e14e5877d204423bca338a1076
I20260807 15:11:27.109295 32170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:27.098037 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.194.1:48377 (local address 127.29.194.62:45747)
0807 15:11:27.098107 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:27.098109 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:27.098306 (+ 197us) server_negotiation.cc:407] Connection header received
0807 15:11:27.098834 (+ 528us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:27.098835 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:27.098849 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:27.098931 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:11:27.099888 (+ 957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:27.100649 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:27.102576 (+ 1927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:27.102920 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:27.103026 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:27.106464 (+ 3438us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:11:27.106466 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:11:27.106466 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:11:27.106475 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:11:27.107899 (+ 1424us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:27.108318 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:27.108319 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:27.108319 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:27.108383 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:27.108614 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:27.108615 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:27.108616 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:27.108738 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:11:27.108795 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0807 15:11:27.109091 (+ 296us) server_negotiation.cc:299] Negotiation successful
0807 15:11:27.109202 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260807 15:11:27.109509 32244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:27.097985 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:45747 (local address 127.29.194.1:48377)
0807 15:11:27.098165 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0807 15:11:27.098208 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0807 15:11:27.098718 (+ 510us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:11:27.099011 (+ 293us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:11:27.099015 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:11:27.099457 (+ 442us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:11:27.099765 (+ 308us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:27.099772 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:27.100838 (+ 1066us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:27.100840 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:27.102426 (+ 1586us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:27.102432 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:27.103080 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:27.103082 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:27.103314 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:27.104254 (+ 940us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:11:27.104265 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:11:27.106335 (+ 2070us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:11:27.107994 (+ 1659us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:11:27.108004 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:11:27.108006 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:11:27.108247 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:11:27.108440 (+ 193us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:11:27.108441 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:11:27.108441 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:11:27.108558 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:11:27.108794 (+ 236us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:11:27.108802 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:11:27.108963 (+ 161us) client_negotiation.cc:769] Sending connection context
0807 15:11:27.109058 (+ 95us) client_negotiation.cc:240] Negotiation successful
0807 15:11:27.109245 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":95,"threads_started":1}
I20260807 15:11:27.110064 32394 heartbeater.cc:347] Connected to a master server at 127.29.194.62:45747
I20260807 15:11:27.110201 32394 heartbeater.cc:464] Registering TS with master...
I20260807 15:11:27.110452 32394 heartbeater.cc:511] Master 127.29.194.62:45747 requested a full tablet report, sending...
I20260807 15:11:27.111104 32103 ts_manager.cc:194] Registered new tserver with Master: 62c4c4f74c3744ddbc461ef9b6acfad4 (127.29.194.1:35187)
I20260807 15:11:27.112010 32103 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.194.1@KRBTEST.COM'} at 127.29.194.1:48377
WARNING: no policy specified for kudu/127.29.194.2@KRBTEST.COM; defaulting to no policy
I20260807 15:11:27.117321 32404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:27.111136 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:45747 (local address 127.29.194.1:53793)
0807 15:11:27.111394 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0807 15:11:27.111406 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 15:11:27.111495 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:11:27.111647 (+ 152us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:11:27.111648 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:11:27.111872 (+ 224us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:11:27.111938 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:27.111940 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:27.112726 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:27.112727 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:27.113259 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:27.113260 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:27.113677 (+ 417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:27.113677 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:27.113758 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:27.114208 (+ 450us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:11:27.114213 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:11:27.114542 (+ 329us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:11:27.116009 (+ 1467us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:11:27.116011 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:11:27.116011 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:11:27.116185 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:11:27.116636 (+ 451us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:11:27.116637 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:11:27.116637 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:11:27.116715 (+ 78us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:11:27.116978 (+ 263us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:11:27.116979 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:11:27.117033 (+ 54us) client_negotiation.cc:769] Sending connection context
0807 15:11:27.117097 (+ 64us) client_negotiation.cc:240] Negotiation successful
0807 15:11:27.117216 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":231,"thread_start_us":189,"threads_started":1}
I20260807 15:11:27.117316 32170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:27.111197 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.194.1:53793 (local address 127.29.194.62:45747)
0807 15:11:27.111235 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:27.111237 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:27.111469 (+ 232us) server_negotiation.cc:407] Connection header received
0807 15:11:27.111557 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:27.111558 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:27.111568 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:27.111594 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:11:27.112019 (+ 425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:27.112578 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:27.113340 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:27.113602 (+ 262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:27.113658 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:27.114628 (+ 970us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:11:27.114630 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:11:27.114630 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:11:27.114638 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:11:27.115922 (+ 1284us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:27.116248 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:27.116249 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:27.116249 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:27.116550 (+ 301us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:27.116772 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:27.116773 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:27.116774 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:27.116914 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:11:27.116979 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0807 15:11:27.117125 (+ 146us) server_negotiation.cc:299] Negotiation successful
0807 15:11:27.117214 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.194.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.194.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.2.keytab.
Entry for principal kudu/127.29.194.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.2.keytab.
I20260807 15:11:27.137580 30472 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7fZSIL/build/release/bin/kudu
/tmp/dist-test-task7fZSIL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115480851507-30472-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115480851507-30472-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115480851507-30472-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115480851507-30472-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.194.2:0
--local_ip_for_outbound_sockets=127.29.194.2
--webserver_interface=127.29.194.2
--webserver_port=0
--tserver_master_addrs=127.29.194.62:45747
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.2.keytab
--principal=kudu/127.29.194.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.194.20:36373
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115480851507-30472-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115480851507-30472-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115480851507-30472-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:11:27.230969 32406 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:11:27.231221 32406 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:11:27.231247 32406 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:11:27.232534 32406 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:11:27.232586 32406 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:11:27.232607 32406 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 15:11:27.232625 32406 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.194.2
I20260807 15:11:27.234264 32406 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.194.20:36373
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115480851507-30472-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115480851507-30472-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.2.keytab
--principal=kudu/127.29.194.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.194.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115480851507-30472-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.194.2
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.29.194.62:45747
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32406
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.194.2
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115480851507-30472-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:11:27.234525 32406 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:11:27.234741 32406 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:11:27.240701 32417 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:11:27.240702 32416 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:11:27.240701 32419 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:11:27.241016 32406 server_base.cc:1034] running on GCE node
I20260807 15:11:27.241361 32406 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:11:27.241686 32406 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:11:27.242875 32406 hybrid_clock.cc:648] HybridClock initialized: now 1786115487242836 us; error 57 us; skew 500 ppm
Aug 07 15:11:27 dist-test-slave-42z9 krb5kdc[32035](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115487, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.194.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:11:27.246315 32406 init.cc:377] Logged in from keytab as kudu/127.29.194.2@KRBTEST.COM (short username kudu)
I20260807 15:11:27.246901 32406 webserver.cc:460] Webserver started at http://127.29.194.2:42673/ using document root <none> and password file <none>
I20260807 15:11:27.247146 32406 fs_manager.cc:359] Metadata directory not provided
I20260807 15:11:27.247202 32406 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:11:27.247329 32406 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:11:27.248453 32406 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115480851507-30472-0/minicluster-data/ts-1/data/instance:
uuid: "4a80a1ab47ee4291bc69d1719068abed"
format_stamp: "Formatted at 2026-08-07 15:11:27 on dist-test-slave-42z9"
server_key: "8f4c2da24b849d5504666c3892584116"
server_key_iv: "3344491bdc06297f74342f83d3634fd8"
server_key_version: "encryptionkey@0"
I20260807 15:11:27.248828 32406 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115480851507-30472-0/minicluster-data/ts-1/wal/instance:
uuid: "4a80a1ab47ee4291bc69d1719068abed"
format_stamp: "Formatted at 2026-08-07 15:11:27 on dist-test-slave-42z9"
server_key: "8f4c2da24b849d5504666c3892584116"
server_key_iv: "3344491bdc06297f74342f83d3634fd8"
server_key_version: "encryptionkey@0"
I20260807 15:11:27.251071 32406 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.004s
I20260807 15:11:27.251948 32430 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:11:27.252384 32406 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 15:11:27.252530 32406 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115480851507-30472-0/minicluster-data/ts-1/data,/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115480851507-30472-0/minicluster-data/ts-1/wal
uuid: "4a80a1ab47ee4291bc69d1719068abed"
format_stamp: "Formatted at 2026-08-07 15:11:27 on dist-test-slave-42z9"
server_key: "8f4c2da24b849d5504666c3892584116"
server_key_iv: "3344491bdc06297f74342f83d3634fd8"
server_key_version: "encryptionkey@0"
I20260807 15:11:27.252628 32406 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115480851507-30472-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115480851507-30472-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115480851507-30472-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:11:27.269055 32406 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:11:27.271396 32406 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:11:27.271553 32406 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:11:27.272145 32406 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:11:27.272198 32406 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:11:27.272224 32406 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:11:27.272238 32406 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:11:27.279173 32406 rpc_server.cc:307] RPC server started. Bound to: 127.29.194.2:37143
Aug 07 15:11:27 dist-test-slave-42z9 krb5kdc[32035](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115487, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.194.2@KRBTEST.COM for kudu/127.29.194.62@KRBTEST.COM
I20260807 15:11:27.279295 32597 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.194.2:37143 every 8 connection(s)
I20260807 15:11:27.279600 32406 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115480851507-30472-0/minicluster-data/ts-1/data/info.pb
I20260807 15:11:27.282399 32170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:27.272702 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.194.2:44199 (local address 127.29.194.62:45747)
0807 15:11:27.272751 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:27.272752 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:27.272995 (+ 243us) server_negotiation.cc:407] Connection header received
0807 15:11:27.273527 (+ 532us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:27.273528 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:27.273543 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:27.273622 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:11:27.274411 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:27.275201 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:27.276736 (+ 1535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:27.277053 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:27.277136 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:27.279822 (+ 2686us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:11:27.279824 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:11:27.279825 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:11:27.279834 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:11:27.281048 (+ 1214us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:27.281476 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:27.281476 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:27.281477 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:27.281531 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:27.281755 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:27.281756 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:27.281757 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:27.281887 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:11:27.281945 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0807 15:11:27.282219 (+ 274us) server_negotiation.cc:299] Negotiation successful
0807 15:11:27.282307 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260807 15:11:27.282584 32440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:27.272696 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:45747 (local address 127.29.194.2:44199)
0807 15:11:27.272862 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0807 15:11:27.272899 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0807 15:11:27.273390 (+ 491us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:11:27.273684 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:11:27.273688 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:11:27.274007 (+ 319us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:11:27.274307 (+ 300us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:27.274313 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:27.275308 (+ 995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:27.275309 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:27.276639 (+ 1330us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:27.276642 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:27.277149 (+ 507us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:27.277150 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:27.277290 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:27.277880 (+ 590us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:11:27.277892 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:11:27.279719 (+ 1827us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:11:27.281134 (+ 1415us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:11:27.281142 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:11:27.281144 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:11:27.281376 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:11:27.281589 (+ 213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:11:27.281590 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:11:27.281590 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:11:27.281700 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:11:27.281950 (+ 250us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:11:27.281957 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:11:27.282097 (+ 140us) client_negotiation.cc:769] Sending connection context
0807 15:11:27.282188 (+ 91us) client_negotiation.cc:240] Negotiation successful
0807 15:11:27.282362 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":94,"threads_started":1}
I20260807 15:11:27.283336 32598 heartbeater.cc:347] Connected to a master server at 127.29.194.62:45747
I20260807 15:11:27.283449 32598 heartbeater.cc:464] Registering TS with master...
I20260807 15:11:27.283654 32598 heartbeater.cc:511] Master 127.29.194.62:45747 requested a full tablet report, sending...
I20260807 15:11:27.284196 32103 ts_manager.cc:194] Registered new tserver with Master: 4a80a1ab47ee4291bc69d1719068abed (127.29.194.2:37143)
I20260807 15:11:27.284792 32103 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.194.2@KRBTEST.COM'} at 127.29.194.2:44199
I20260807 15:11:27.285197 30472 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7fZSIL/build/release/bin/kudu as pid 32406
I20260807 15:11:27.285310 30472 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115480851507-30472-0/minicluster-data/ts-1/wal/instance
I20260807 15:11:27.285575 30472 external_mini_cluster.cc:1428] Setting key a566078861aeb77f2e4c4612b8726b3c
I20260807 15:11:27.292040 32607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:27.284466 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:45747 (local address 127.29.194.2:38231)
0807 15:11:27.284649 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0807 15:11:27.284659 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 15:11:27.284722 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:11:27.284853 (+ 131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:11:27.284854 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:11:27.285080 (+ 226us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:11:27.285139 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:27.285141 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:27.286290 (+ 1149us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:27.286292 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:27.287025 (+ 733us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:27.287029 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:27.287461 (+ 432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:27.287463 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:27.287549 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:27.288177 (+ 628us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:11:27.288183 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:11:27.288762 (+ 579us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:11:27.290797 (+ 2035us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:11:27.290798 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:11:27.290799 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:11:27.291034 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:11:27.291257 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:11:27.291258 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:11:27.291258 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:11:27.291314 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:11:27.291685 (+ 371us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:11:27.291685 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:11:27.291745 (+ 60us) client_negotiation.cc:769] Sending connection context
0807 15:11:27.291815 (+ 70us) client_negotiation.cc:240] Negotiation successful
0807 15:11:27.291944 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":70,"threads_started":1}
I20260807 15:11:27.292040 32170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:27.284573 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.194.2:38231 (local address 127.29.194.62:45747)
0807 15:11:27.284617 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:27.284619 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:27.284718 (+ 99us) server_negotiation.cc:407] Connection header received
0807 15:11:27.284779 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:27.284780 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:27.284790 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:27.284833 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:11:27.285205 (+ 372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:27.285971 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:27.287149 (+ 1178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:27.287375 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:27.287445 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:27.288968 (+ 1523us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:11:27.288970 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:11:27.288970 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:11:27.288978 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:11:27.290688 (+ 1710us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:27.291115 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:27.291117 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:27.291117 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:27.291191 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:27.291363 (+ 172us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:27.291363 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:27.291364 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:27.291581 (+ 217us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:11:27.291657 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0807 15:11:27.291851 (+ 194us) server_negotiation.cc:299] Negotiation successful
0807 15:11:27.291951 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
WARNING: no policy specified for kudu/127.29.194.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.194.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.194.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.3.keytab.
Entry for principal kudu/127.29.194.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.3.keytab.
I20260807 15:11:27.320606 30472 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7fZSIL/build/release/bin/kudu
/tmp/dist-test-task7fZSIL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115480851507-30472-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115480851507-30472-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115480851507-30472-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115480851507-30472-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.194.3:0
--local_ip_for_outbound_sockets=127.29.194.3
--webserver_interface=127.29.194.3
--webserver_port=0
--tserver_master_addrs=127.29.194.62:45747
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.3.keytab
--principal=kudu/127.29.194.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.194.20:36373
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115480851507-30472-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115480851507-30472-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115480851507-30472-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:11:27.413686 32611 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:11:27.413939 32611 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:11:27.413968 32611 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:11:27.415256 32611 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:11:27.415305 32611 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:11:27.415326 32611 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 15:11:27.415343 32611 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.194.3
I20260807 15:11:27.416929 32611 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.194.20:36373
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115480851507-30472-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115480851507-30472-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.3.keytab
--principal=kudu/127.29.194.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.194.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115480851507-30472-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.194.3
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.29.194.62:45747
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32611
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.194.3
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115480851507-30472-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:11:27.417174 32611 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:11:27.417378 32611 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:11:27.423210 32620 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:11:27.423206 32617 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:11:27.423187 32618 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:11:27.423172 32611 server_base.cc:1034] running on GCE node
I20260807 15:11:27.423619 32611 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:11:27.423893 32611 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:11:27.425081 32611 hybrid_clock.cc:648] HybridClock initialized: now 1786115487425043 us; error 69 us; skew 500 ppm
Aug 07 15:11:27 dist-test-slave-42z9 krb5kdc[32035](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115487, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.194.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:11:27.427981 32611 init.cc:377] Logged in from keytab as kudu/127.29.194.3@KRBTEST.COM (short username kudu)
I20260807 15:11:27.428510 32611 webserver.cc:460] Webserver started at http://127.29.194.3:35537/ using document root <none> and password file <none>
I20260807 15:11:27.428755 32611 fs_manager.cc:359] Metadata directory not provided
I20260807 15:11:27.428808 32611 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:11:27.428947 32611 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:11:27.430135 32611 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115480851507-30472-0/minicluster-data/ts-2/data/instance:
uuid: "4e1cf8eab0004fa8a9101887482ef407"
format_stamp: "Formatted at 2026-08-07 15:11:27 on dist-test-slave-42z9"
server_key: "03122a3632527dd1d887a690f36b3b6f"
server_key_iv: "81111c1b6f1112d879c9a7ebac7c9370"
server_key_version: "encryptionkey@0"
I20260807 15:11:27.430500 32611 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115480851507-30472-0/minicluster-data/ts-2/wal/instance:
uuid: "4e1cf8eab0004fa8a9101887482ef407"
format_stamp: "Formatted at 2026-08-07 15:11:27 on dist-test-slave-42z9"
server_key: "03122a3632527dd1d887a690f36b3b6f"
server_key_iv: "81111c1b6f1112d879c9a7ebac7c9370"
server_key_version: "encryptionkey@0"
I20260807 15:11:27.432085 32611 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 15:11:27.433326 32629 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:11:27.433586 32611 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:11:27.433691 32611 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115480851507-30472-0/minicluster-data/ts-2/data,/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115480851507-30472-0/minicluster-data/ts-2/wal
uuid: "4e1cf8eab0004fa8a9101887482ef407"
format_stamp: "Formatted at 2026-08-07 15:11:27 on dist-test-slave-42z9"
server_key: "03122a3632527dd1d887a690f36b3b6f"
server_key_iv: "81111c1b6f1112d879c9a7ebac7c9370"
server_key_version: "encryptionkey@0"
I20260807 15:11:27.433766 32611 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115480851507-30472-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115480851507-30472-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115480851507-30472-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:11:27.474750 32611 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:11:27.477911 32611 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:11:27.478060 32611 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:11:27.478696 32611 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:11:27.478750 32611 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:11:27.478775 32611 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:11:27.478785 32611 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:11:27.486692 32611 rpc_server.cc:307] RPC server started. Bound to: 127.29.194.3:34479
I20260807 15:11:27.486809 345 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.194.3:34479 every 8 connection(s)
I20260807 15:11:27.487110 32611 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115480851507-30472-0/minicluster-data/ts-2/data/info.pb
Aug 07 15:11:27 dist-test-slave-42z9 krb5kdc[32035](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115487, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.194.3@KRBTEST.COM for kudu/127.29.194.62@KRBTEST.COM
I20260807 15:11:27.488888 30472 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7fZSIL/build/release/bin/kudu as pid 32611
I20260807 15:11:27.488983 30472 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115480851507-30472-0/minicluster-data/ts-2/wal/instance
I20260807 15:11:27.489164 30472 external_mini_cluster.cc:1428] Setting key 2938001c187857fbf2ad8cbad9411145
I20260807 15:11:27.491089 32170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:27.479351 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.194.3:52835 (local address 127.29.194.62:45747)
0807 15:11:27.479436 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:27.479439 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:27.479684 (+ 245us) server_negotiation.cc:407] Connection header received
0807 15:11:27.480433 (+ 749us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:27.480435 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:27.480453 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:27.480575 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:11:27.481621 (+ 1046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:27.482476 (+ 855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:27.484261 (+ 1785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:27.484593 (+ 332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:27.484680 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:27.487749 (+ 3069us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:11:27.487751 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:11:27.487752 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:11:27.487760 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:11:27.489657 (+ 1897us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:27.490084 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:27.490085 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:27.490086 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:27.490150 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:27.490405 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:27.490405 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:27.490406 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:27.490542 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:11:27.490603 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0807 15:11:27.490892 (+ 289us) server_negotiation.cc:299] Negotiation successful
0807 15:11:27.490999 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260807 15:11:27.491271 32641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:27.479278 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:45747 (local address 127.29.194.3:52835)
0807 15:11:27.479528 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0807 15:11:27.479575 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0807 15:11:27.480250 (+ 675us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:11:27.480680 (+ 430us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:11:27.480688 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:11:27.481123 (+ 435us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:11:27.481511 (+ 388us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:27.481518 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:27.482616 (+ 1098us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:27.482619 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:27.484112 (+ 1493us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:27.484120 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:27.484701 (+ 581us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:27.484702 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:27.484893 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:27.485735 (+ 842us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:11:27.485749 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:11:27.487615 (+ 1866us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:11:27.489731 (+ 2116us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:11:27.489743 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:11:27.489746 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:11:27.490009 (+ 263us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:11:27.490207 (+ 198us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:11:27.490208 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:11:27.490209 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:11:27.490343 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:11:27.490602 (+ 259us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:11:27.490610 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:11:27.490761 (+ 151us) client_negotiation.cc:769] Sending connection context
0807 15:11:27.490859 (+ 98us) client_negotiation.cc:240] Negotiation successful
0807 15:11:27.491041 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":127,"threads_started":1}
I20260807 15:11:27.491746 347 heartbeater.cc:347] Connected to a master server at 127.29.194.62:45747
I20260807 15:11:27.491848 347 heartbeater.cc:464] Registering TS with master...
I20260807 15:11:27.492046 347 heartbeater.cc:511] Master 127.29.194.62:45747 requested a full tablet report, sending...
I20260807 15:11:27.492427 32105 ts_manager.cc:194] Registered new tserver with Master: 4e1cf8eab0004fa8a9101887482ef407 (127.29.194.3:34479)
I20260807 15:11:27.492836 32105 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.194.3@KRBTEST.COM'} at 127.29.194.3:52835
I20260807 15:11:27.499235 32170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:27.492901 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.194.3:36985 (local address 127.29.194.62:45747)
0807 15:11:27.492944 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:27.492946 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:27.493067 (+ 121us) server_negotiation.cc:407] Connection header received
0807 15:11:27.493135 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:27.493136 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:27.493145 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:27.493188 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:11:27.493538 (+ 350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:27.494079 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:27.494761 (+ 682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:27.494965 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:27.495020 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:27.496274 (+ 1254us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:11:27.496276 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:11:27.496276 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:11:27.496287 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:11:27.497941 (+ 1654us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:27.498384 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:27.498385 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:27.498385 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:27.498447 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:27.498624 (+ 177us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:27.498625 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:27.498625 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:27.498775 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:11:27.498837 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0807 15:11:27.498990 (+ 153us) server_negotiation.cc:299] Negotiation successful
0807 15:11:27.499099 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260807 15:11:27.499238 359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:27.492826 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:45747 (local address 127.29.194.3:36985)
0807 15:11:27.493014 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0807 15:11:27.493025 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 15:11:27.493081 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:11:27.493191 (+ 110us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:11:27.493192 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:11:27.493384 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:11:27.493468 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:27.493471 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:27.494181 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:27.494182 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:27.494688 (+ 506us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:27.494689 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:27.495024 (+ 335us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:27.495025 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:27.495101 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:27.495611 (+ 510us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:11:27.495619 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:11:27.496139 (+ 520us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:11:27.498060 (+ 1921us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:11:27.498061 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:11:27.498062 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:11:27.498296 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:11:27.498505 (+ 209us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:11:27.498506 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:11:27.498506 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:11:27.498573 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:11:27.498832 (+ 259us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:11:27.498833 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:11:27.498899 (+ 66us) client_negotiation.cc:769] Sending connection context
0807 15:11:27.498959 (+ 60us) client_negotiation.cc:240] Negotiation successful
0807 15:11:27.499103 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":64,"threads_started":1}
I20260807 15:11:27.501355 30472 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 07 15:11:27 dist-test-slave-42z9 krb5kdc[32035](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115487, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 07 15:11:27 dist-test-slave-42z9 krb5kdc[32035](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115487, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.29.194.62@KRBTEST.COM
I20260807 15:11:27.530220 32170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:27.521726 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43848 (local address 127.29.194.62:45747)
0807 15:11:27.521815 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:27.521817 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:27.521835 (+ 18us) server_negotiation.cc:407] Connection header received
0807 15:11:27.521916 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:27.521917 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:27.521929 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:27.521994 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:11:27.522825 (+ 831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:27.523395 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:27.524323 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:27.524553 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:27.524626 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:27.527220 (+ 2594us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:11:27.527223 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:11:27.527223 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:11:27.527233 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:11:27.528689 (+ 1456us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:27.529247 (+ 558us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:27.529248 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:27.529249 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:27.529316 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:27.529650 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:27.529651 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:27.529651 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:27.529791 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:11:27.529887 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0807 15:11:27.530061 (+ 174us) server_negotiation.cc:299] Negotiation successful
0807 15:11:27.530143 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260807 15:11:27.532168 32105 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43848:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 15:11:27.534438 32105 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 15:11:27.542316 378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:27.537962 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34038 (local address 127.29.194.3:34479)
0807 15:11:27.538156 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:27.538161 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:27.538342 (+ 181us) server_negotiation.cc:407] Connection header received
0807 15:11:27.538392 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:27.538395 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:27.538411 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:27.538483 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:11:27.538719 (+ 236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:27.539940 (+ 1221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:27.541260 (+ 1320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:27.541880 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:27.541951 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:27.542021 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0807 15:11:27.542164 (+ 143us) server_negotiation.cc:299] Negotiation successful
0807 15:11:27.542205 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":160,"thread_start_us":88,"threads_started":1}
I20260807 15:11:27.542301 376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:27.537886 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.3:34479 (local address 127.0.0.1:34038)
0807 15:11:27.538118 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0807 15:11:27.538163 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0807 15:11:27.538251 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:11:27.538562 (+ 311us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:11:27.538565 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:11:27.538567 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:11:27.538659 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:27.538661 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:27.540052 (+ 1391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:27.540053 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:27.541168 (+ 1115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:27.541170 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:27.541963 (+ 793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:27.541964 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:27.542055 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:27.542070 (+ 15us) client_negotiation.cc:769] Sending connection context
0807 15:11:27.542140 (+ 70us) client_negotiation.cc:240] Negotiation successful
0807 15:11:27.542201 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":203,"mutex_wait_us":34,"spinlock_wait_cycles":768,"thread_start_us":130,"threads_started":1}
I20260807 15:11:27.542470 380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:27.538016 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42626 (local address 127.29.194.2:37143)
0807 15:11:27.538781 (+ 765us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:27.538786 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:27.538795 (+ 9us) server_negotiation.cc:407] Connection header received
0807 15:11:27.538830 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:27.538843 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:27.538857 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:27.538921 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:11:27.539028 (+ 107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:27.539739 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:27.541243 (+ 1504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:27.541976 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:27.542029 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:27.542076 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0807 15:11:27.542326 (+ 250us) server_negotiation.cc:299] Negotiation successful
0807 15:11:27.542369 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":731,"thread_start_us":129,"threads_started":1}
I20260807 15:11:27.542485 382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:27.538338 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34140 (local address 127.29.194.1:35187)
0807 15:11:27.538517 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:27.538521 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:27.538531 (+ 10us) server_negotiation.cc:407] Connection header received
0807 15:11:27.538574 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:27.538576 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:27.538590 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:27.538669 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:11:27.538802 (+ 133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:27.539969 (+ 1167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:27.541131 (+ 1162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:27.541928 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:27.542004 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:27.542067 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0807 15:11:27.542352 (+ 285us) server_negotiation.cc:299] Negotiation successful
0807 15:11:27.542394 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":150,"thread_start_us":79,"threads_started":1}
I20260807 15:11:27.542681 381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:27.537887 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.2:37143 (local address 127.0.0.1:42626)
0807 15:11:27.538274 (+ 387us) negotiation.cc:107] Waiting for socket to connect
0807 15:11:27.538283 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0807 15:11:27.538350 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:11:27.538931 (+ 581us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:11:27.538932 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:11:27.538934 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:11:27.538977 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:27.538979 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:27.539826 (+ 847us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:27.539827 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:27.541145 (+ 1318us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:27.541148 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:27.542114 (+ 966us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:27.542115 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:27.542191 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:27.542197 (+ 6us) client_negotiation.cc:769] Sending connection context
0807 15:11:27.542245 (+ 48us) client_negotiation.cc:240] Negotiation successful
0807 15:11:27.542616 (+ 371us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":369,"mutex_wait_us":316,"thread_start_us":100,"threads_started":1}
I20260807 15:11:27.542840 377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:27.538150 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.1:35187 (local address 127.0.0.1:34140)
0807 15:11:27.538457 (+ 307us) negotiation.cc:107] Waiting for socket to connect
0807 15:11:27.538466 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0807 15:11:27.538507 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:11:27.538667 (+ 160us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:11:27.538668 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:11:27.538670 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:11:27.538734 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:27.538736 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:27.540062 (+ 1326us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:27.540063 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:27.541035 (+ 972us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:27.541038 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:27.542116 (+ 1078us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:27.542117 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:27.542191 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:27.542197 (+ 6us) client_negotiation.cc:769] Sending connection context
0807 15:11:27.542245 (+ 48us) client_negotiation.cc:240] Negotiation successful
0807 15:11:27.542306 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":288,"mutex_wait_us":8,"thread_start_us":72,"threads_started":1}
I20260807 15:11:27.543442 32710 tablet_service.cc:1467] Processing CreateTablet for tablet 855add92bb67414cb778f98cd6c17dd8 (DEFAULT_TABLE table=test-table [id=47bc612cd8f942669098b92857c70198]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:11:27.543442 32499 tablet_service.cc:1467] Processing CreateTablet for tablet 855add92bb67414cb778f98cd6c17dd8 (DEFAULT_TABLE table=test-table [id=47bc612cd8f942669098b92857c70198]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:11:27.543573 32297 tablet_service.cc:1467] Processing CreateTablet for tablet 855add92bb67414cb778f98cd6c17dd8 (DEFAULT_TABLE table=test-table [id=47bc612cd8f942669098b92857c70198]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:11:27.543871 32710 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 855add92bb67414cb778f98cd6c17dd8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:11:27.543871 32499 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 855add92bb67414cb778f98cd6c17dd8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:11:27.543913 32297 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 855add92bb67414cb778f98cd6c17dd8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:11:27.547194 383 tablet_bootstrap.cc:492] T 855add92bb67414cb778f98cd6c17dd8 P 62c4c4f74c3744ddbc461ef9b6acfad4: Bootstrap starting.
I20260807 15:11:27.547907 383 tablet_bootstrap.cc:654] T 855add92bb67414cb778f98cd6c17dd8 P 62c4c4f74c3744ddbc461ef9b6acfad4: Neither blocks nor log segments found. Creating new log.
I20260807 15:11:27.548239 383 log.cc:824] T 855add92bb67414cb778f98cd6c17dd8 P 62c4c4f74c3744ddbc461ef9b6acfad4: Log is configured to *not* fsync() on all Append() calls
I20260807 15:11:27.548549 386 tablet_bootstrap.cc:492] T 855add92bb67414cb778f98cd6c17dd8 P 4a80a1ab47ee4291bc69d1719068abed: Bootstrap starting.
I20260807 15:11:27.548599 385 tablet_bootstrap.cc:492] T 855add92bb67414cb778f98cd6c17dd8 P 4e1cf8eab0004fa8a9101887482ef407: Bootstrap starting.
I20260807 15:11:27.549258 386 tablet_bootstrap.cc:654] T 855add92bb67414cb778f98cd6c17dd8 P 4a80a1ab47ee4291bc69d1719068abed: Neither blocks nor log segments found. Creating new log.
I20260807 15:11:27.549576 385 tablet_bootstrap.cc:654] T 855add92bb67414cb778f98cd6c17dd8 P 4e1cf8eab0004fa8a9101887482ef407: Neither blocks nor log segments found. Creating new log.
I20260807 15:11:27.549662 386 log.cc:824] T 855add92bb67414cb778f98cd6c17dd8 P 4a80a1ab47ee4291bc69d1719068abed: Log is configured to *not* fsync() on all Append() calls
I20260807 15:11:27.550136 383 tablet_bootstrap.cc:492] T 855add92bb67414cb778f98cd6c17dd8 P 62c4c4f74c3744ddbc461ef9b6acfad4: No bootstrap required, opened a new log
I20260807 15:11:27.550218 383 ts_tablet_manager.cc:1397] T 855add92bb67414cb778f98cd6c17dd8 P 62c4c4f74c3744ddbc461ef9b6acfad4: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260807 15:11:27.550472 386 tablet_bootstrap.cc:492] T 855add92bb67414cb778f98cd6c17dd8 P 4a80a1ab47ee4291bc69d1719068abed: No bootstrap required, opened a new log
I20260807 15:11:27.550544 386 ts_tablet_manager.cc:1397] T 855add92bb67414cb778f98cd6c17dd8 P 4a80a1ab47ee4291bc69d1719068abed: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260807 15:11:27.552277 383 raft_consensus.cc:348] T 855add92bb67414cb778f98cd6c17dd8 P 62c4c4f74c3744ddbc461ef9b6acfad4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e1cf8eab0004fa8a9101887482ef407" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 34479 } } peers { permanent_uuid: "4a80a1ab47ee4291bc69d1719068abed" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 37143 } } peers { permanent_uuid: "62c4c4f74c3744ddbc461ef9b6acfad4" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 35187 } }
I20260807 15:11:27.552481 383 raft_consensus.cc:374] T 855add92bb67414cb778f98cd6c17dd8 P 62c4c4f74c3744ddbc461ef9b6acfad4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:11:27.552536 383 raft_consensus.cc:729] T 855add92bb67414cb778f98cd6c17dd8 P 62c4c4f74c3744ddbc461ef9b6acfad4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 62c4c4f74c3744ddbc461ef9b6acfad4, State: Initialized, Role: FOLLOWER
I20260807 15:11:27.552486 386 raft_consensus.cc:348] T 855add92bb67414cb778f98cd6c17dd8 P 4a80a1ab47ee4291bc69d1719068abed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e1cf8eab0004fa8a9101887482ef407" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 34479 } } peers { permanent_uuid: "4a80a1ab47ee4291bc69d1719068abed" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 37143 } } peers { permanent_uuid: "62c4c4f74c3744ddbc461ef9b6acfad4" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 35187 } }
I20260807 15:11:27.552625 386 raft_consensus.cc:374] T 855add92bb67414cb778f98cd6c17dd8 P 4a80a1ab47ee4291bc69d1719068abed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:11:27.552665 386 raft_consensus.cc:729] T 855add92bb67414cb778f98cd6c17dd8 P 4a80a1ab47ee4291bc69d1719068abed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4a80a1ab47ee4291bc69d1719068abed, State: Initialized, Role: FOLLOWER
I20260807 15:11:27.552666 383 consensus_queue.cc:260] T 855add92bb67414cb778f98cd6c17dd8 P 62c4c4f74c3744ddbc461ef9b6acfad4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e1cf8eab0004fa8a9101887482ef407" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 34479 } } peers { permanent_uuid: "4a80a1ab47ee4291bc69d1719068abed" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 37143 } } peers { permanent_uuid: "62c4c4f74c3744ddbc461ef9b6acfad4" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 35187 } }
I20260807 15:11:27.552772 386 consensus_queue.cc:260] T 855add92bb67414cb778f98cd6c17dd8 P 4a80a1ab47ee4291bc69d1719068abed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e1cf8eab0004fa8a9101887482ef407" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 34479 } } peers { permanent_uuid: "4a80a1ab47ee4291bc69d1719068abed" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 37143 } } peers { permanent_uuid: "62c4c4f74c3744ddbc461ef9b6acfad4" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 35187 } }
I20260807 15:11:27.553022 386 ts_tablet_manager.cc:1428] T 855add92bb67414cb778f98cd6c17dd8 P 4a80a1ab47ee4291bc69d1719068abed: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260807 15:11:27.553047 32394 heartbeater.cc:503] Master 127.29.194.62:45747 was elected leader, sending a full tablet report...
I20260807 15:11:27.553092 32598 heartbeater.cc:503] Master 127.29.194.62:45747 was elected leader, sending a full tablet report...
I20260807 15:11:27.552996 383 ts_tablet_manager.cc:1428] T 855add92bb67414cb778f98cd6c17dd8 P 62c4c4f74c3744ddbc461ef9b6acfad4: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260807 15:11:27.553709 385 log.cc:824] T 855add92bb67414cb778f98cd6c17dd8 P 4e1cf8eab0004fa8a9101887482ef407: Log is configured to *not* fsync() on all Append() calls
I20260807 15:11:27.554563 385 tablet_bootstrap.cc:492] T 855add92bb67414cb778f98cd6c17dd8 P 4e1cf8eab0004fa8a9101887482ef407: No bootstrap required, opened a new log
I20260807 15:11:27.554636 385 ts_tablet_manager.cc:1397] T 855add92bb67414cb778f98cd6c17dd8 P 4e1cf8eab0004fa8a9101887482ef407: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.002s
I20260807 15:11:27.556408 385 raft_consensus.cc:348] T 855add92bb67414cb778f98cd6c17dd8 P 4e1cf8eab0004fa8a9101887482ef407 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e1cf8eab0004fa8a9101887482ef407" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 34479 } } peers { permanent_uuid: "4a80a1ab47ee4291bc69d1719068abed" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 37143 } } peers { permanent_uuid: "62c4c4f74c3744ddbc461ef9b6acfad4" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 35187 } }
I20260807 15:11:27.556617 385 raft_consensus.cc:374] T 855add92bb67414cb778f98cd6c17dd8 P 4e1cf8eab0004fa8a9101887482ef407 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:11:27.556660 385 raft_consensus.cc:729] T 855add92bb67414cb778f98cd6c17dd8 P 4e1cf8eab0004fa8a9101887482ef407 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e1cf8eab0004fa8a9101887482ef407, State: Initialized, Role: FOLLOWER
I20260807 15:11:27.556794 385 consensus_queue.cc:260] T 855add92bb67414cb778f98cd6c17dd8 P 4e1cf8eab0004fa8a9101887482ef407 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e1cf8eab0004fa8a9101887482ef407" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 34479 } } peers { permanent_uuid: "4a80a1ab47ee4291bc69d1719068abed" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 37143 } } peers { permanent_uuid: "62c4c4f74c3744ddbc461ef9b6acfad4" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 35187 } }
I20260807 15:11:27.557091 385 ts_tablet_manager.cc:1428] T 855add92bb67414cb778f98cd6c17dd8 P 4e1cf8eab0004fa8a9101887482ef407: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260807 15:11:27.557166 347 heartbeater.cc:503] Master 127.29.194.62:45747 was elected leader, sending a full tablet report...
W20260807 15:11:27.606195 32395 tablet.cc:2369] T 855add92bb67414cb778f98cd6c17dd8 P 62c4c4f74c3744ddbc461ef9b6acfad4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 15:11:27.647960 392 raft_consensus.cc:482] T 855add92bb67414cb778f98cd6c17dd8 P 62c4c4f74c3744ddbc461ef9b6acfad4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 15:11:27.648132 392 raft_consensus.cc:504] T 855add92bb67414cb778f98cd6c17dd8 P 62c4c4f74c3744ddbc461ef9b6acfad4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e1cf8eab0004fa8a9101887482ef407" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 34479 } } peers { permanent_uuid: "4a80a1ab47ee4291bc69d1719068abed" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 37143 } } peers { permanent_uuid: "62c4c4f74c3744ddbc461ef9b6acfad4" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 35187 } }
I20260807 15:11:27.648530 392 leader_election.cc:288] T 855add92bb67414cb778f98cd6c17dd8 P 62c4c4f74c3744ddbc461ef9b6acfad4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4e1cf8eab0004fa8a9101887482ef407 (127.29.194.3:34479), 4a80a1ab47ee4291bc69d1719068abed (127.29.194.2:37143)
I20260807 15:11:27.651760 380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:27.648998 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.194.1:42899 (local address 127.29.194.2:37143)
0807 15:11:27.649049 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:27.649051 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:27.649181 (+ 130us) server_negotiation.cc:407] Connection header received
0807 15:11:27.649261 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:27.649262 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:27.649273 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:27.649324 (+ 51us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:11:27.649530 (+ 206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:27.650039 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:27.651040 (+ 1001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:27.651418 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:27.651487 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:27.651496 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0807 15:11:27.651643 (+ 147us) server_negotiation.cc:299] Negotiation successful
0807 15:11:27.651683 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260807 15:11:27.651760 378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:27.648831 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.194.1:60991 (local address 127.29.194.3:34479)
0807 15:11:27.648884 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:27.648886 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:27.649029 (+ 143us) server_negotiation.cc:407] Connection header received
0807 15:11:27.649130 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:27.649132 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:27.649142 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:27.649192 (+ 50us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:11:27.649351 (+ 159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:27.650068 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:27.651060 (+ 992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:27.651433 (+ 373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:27.651499 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:27.651508 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0807 15:11:27.651647 (+ 139us) server_negotiation.cc:299] Negotiation successful
0807 15:11:27.651687 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260807 15:11:27.651813 398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:27.648919 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.2:37143 (local address 127.29.194.1:42899)
0807 15:11:27.649100 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0807 15:11:27.649112 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 15:11:27.649196 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:11:27.649348 (+ 152us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:11:27.649349 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:11:27.649354 (+ 5us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:11:27.649470 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:27.649473 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:27.650121 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:27.650122 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:27.650939 (+ 817us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:27.650941 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:27.651495 (+ 554us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:27.651496 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:27.651575 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:27.651579 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 15:11:27.651627 (+ 48us) client_negotiation.cc:240] Negotiation successful
0807 15:11:27.651683 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":157,"mutex_wait_us":18,"thread_start_us":87,"threads_started":1}
I20260807 15:11:27.651815 397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:27.648710 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.3:34479 (local address 127.29.194.1:60991)
0807 15:11:27.648909 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0807 15:11:27.648926 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 15:11:27.649029 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:11:27.649213 (+ 184us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:11:27.649214 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:11:27.649217 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:11:27.649294 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:27.649296 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:27.650215 (+ 919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:27.650216 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:27.650965 (+ 749us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:27.650967 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:27.651516 (+ 549us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:27.651517 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:27.651582 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:27.651586 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 15:11:27.651633 (+ 47us) client_negotiation.cc:240] Negotiation successful
0807 15:11:27.651686 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":91,"threads_started":1}
I20260807 15:11:27.652209 32527 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "855add92bb67414cb778f98cd6c17dd8" candidate_uuid: "62c4c4f74c3744ddbc461ef9b6acfad4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4a80a1ab47ee4291bc69d1719068abed" is_pre_election: true
I20260807 15:11:27.652381 32527 raft_consensus.cc:2454] T 855add92bb67414cb778f98cd6c17dd8 P 4a80a1ab47ee4291bc69d1719068abed [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 62c4c4f74c3744ddbc461ef9b6acfad4 in term 0.
I20260807 15:11:27.652395 32743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "855add92bb67414cb778f98cd6c17dd8" candidate_uuid: "62c4c4f74c3744ddbc461ef9b6acfad4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e1cf8eab0004fa8a9101887482ef407" is_pre_election: true
I20260807 15:11:27.652520 32743 raft_consensus.cc:2454] T 855add92bb67414cb778f98cd6c17dd8 P 4e1cf8eab0004fa8a9101887482ef407 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 62c4c4f74c3744ddbc461ef9b6acfad4 in term 0.
I20260807 15:11:27.652653 32231 leader_election.cc:302] T 855add92bb67414cb778f98cd6c17dd8 P 62c4c4f74c3744ddbc461ef9b6acfad4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4a80a1ab47ee4291bc69d1719068abed, 62c4c4f74c3744ddbc461ef9b6acfad4; no voters:
I20260807 15:11:27.652801 392 raft_consensus.cc:2781] T 855add92bb67414cb778f98cd6c17dd8 P 62c4c4f74c3744ddbc461ef9b6acfad4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 15:11:27.652872 392 raft_consensus.cc:482] T 855add92bb67414cb778f98cd6c17dd8 P 62c4c4f74c3744ddbc461ef9b6acfad4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 15:11:27.652890 392 raft_consensus.cc:3037] T 855add92bb67414cb778f98cd6c17dd8 P 62c4c4f74c3744ddbc461ef9b6acfad4 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:11:27.653692 392 raft_consensus.cc:504] T 855add92bb67414cb778f98cd6c17dd8 P 62c4c4f74c3744ddbc461ef9b6acfad4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e1cf8eab0004fa8a9101887482ef407" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 34479 } } peers { permanent_uuid: "4a80a1ab47ee4291bc69d1719068abed" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 37143 } } peers { permanent_uuid: "62c4c4f74c3744ddbc461ef9b6acfad4" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 35187 } }
I20260807 15:11:27.653826 392 leader_election.cc:288] T 855add92bb67414cb778f98cd6c17dd8 P 62c4c4f74c3744ddbc461ef9b6acfad4 [CANDIDATE]: Term 1 election: Requested vote from peers 4e1cf8eab0004fa8a9101887482ef407 (127.29.194.3:34479), 4a80a1ab47ee4291bc69d1719068abed (127.29.194.2:37143)
I20260807 15:11:27.654024 32743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "855add92bb67414cb778f98cd6c17dd8" candidate_uuid: "62c4c4f74c3744ddbc461ef9b6acfad4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e1cf8eab0004fa8a9101887482ef407"
I20260807 15:11:27.654008 32527 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "855add92bb67414cb778f98cd6c17dd8" candidate_uuid: "62c4c4f74c3744ddbc461ef9b6acfad4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4a80a1ab47ee4291bc69d1719068abed"
I20260807 15:11:27.654101 32527 raft_consensus.cc:3037] T 855add92bb67414cb778f98cd6c17dd8 P 4a80a1ab47ee4291bc69d1719068abed [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:11:27.654102 32743 raft_consensus.cc:3037] T 855add92bb67414cb778f98cd6c17dd8 P 4e1cf8eab0004fa8a9101887482ef407 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:11:27.654857 32527 raft_consensus.cc:2454] T 855add92bb67414cb778f98cd6c17dd8 P 4a80a1ab47ee4291bc69d1719068abed [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 62c4c4f74c3744ddbc461ef9b6acfad4 in term 1.
I20260807 15:11:27.654865 32743 raft_consensus.cc:2454] T 855add92bb67414cb778f98cd6c17dd8 P 4e1cf8eab0004fa8a9101887482ef407 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 62c4c4f74c3744ddbc461ef9b6acfad4 in term 1.
I20260807 15:11:27.655061 32233 leader_election.cc:302] T 855add92bb67414cb778f98cd6c17dd8 P 62c4c4f74c3744ddbc461ef9b6acfad4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e1cf8eab0004fa8a9101887482ef407, 62c4c4f74c3744ddbc461ef9b6acfad4; no voters:
I20260807 15:11:27.655166 392 raft_consensus.cc:2781] T 855add92bb67414cb778f98cd6c17dd8 P 62c4c4f74c3744ddbc461ef9b6acfad4 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:11:27.655321 392 raft_consensus.cc:686] T 855add92bb67414cb778f98cd6c17dd8 P 62c4c4f74c3744ddbc461ef9b6acfad4 [term 1 LEADER]: Becoming Leader. State: Replica: 62c4c4f74c3744ddbc461ef9b6acfad4, State: Running, Role: LEADER
I20260807 15:11:27.655417 392 consensus_queue.cc:237] T 855add92bb67414cb778f98cd6c17dd8 P 62c4c4f74c3744ddbc461ef9b6acfad4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e1cf8eab0004fa8a9101887482ef407" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 34479 } } peers { permanent_uuid: "4a80a1ab47ee4291bc69d1719068abed" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 37143 } } peers { permanent_uuid: "62c4c4f74c3744ddbc461ef9b6acfad4" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 35187 } }
I20260807 15:11:27.656384 32103 catalog_manager.cc:5495] T 855add92bb67414cb778f98cd6c17dd8 P 62c4c4f74c3744ddbc461ef9b6acfad4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 62c4c4f74c3744ddbc461ef9b6acfad4 (127.29.194.1). New cstate: current_term: 1 leader_uuid: "62c4c4f74c3744ddbc461ef9b6acfad4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e1cf8eab0004fa8a9101887482ef407" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 34479 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4a80a1ab47ee4291bc69d1719068abed" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 37143 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "62c4c4f74c3744ddbc461ef9b6acfad4" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 35187 } health_report { overall_health: HEALTHY } } }
I20260807 15:11:27.667107 382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:27.664182 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34142 (local address 127.29.194.1:35187)
0807 15:11:27.664236 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:27.664238 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:27.664247 (+ 9us) server_negotiation.cc:407] Connection header received
0807 15:11:27.664407 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:27.664408 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:27.664426 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:27.664483 (+ 57us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 15:11:27.664616 (+ 133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:27.665130 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:27.666045 (+ 915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:27.666245 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:27.666317 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:27.666705 (+ 388us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 15:11:27.666820 (+ 115us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 15:11:27.666902 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0807 15:11:27.667018 (+ 116us) server_negotiation.cc:299] Negotiation successful
0807 15:11:27.667041 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260807 15:11:27.669690 32743 raft_consensus.cc:1261] T 855add92bb67414cb778f98cd6c17dd8 P 4e1cf8eab0004fa8a9101887482ef407 [term 1 FOLLOWER]: Refusing update from remote peer 62c4c4f74c3744ddbc461ef9b6acfad4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:11:27.669735 32527 raft_consensus.cc:1261] T 855add92bb67414cb778f98cd6c17dd8 P 4a80a1ab47ee4291bc69d1719068abed [term 1 FOLLOWER]: Refusing update from remote peer 62c4c4f74c3744ddbc461ef9b6acfad4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:11:27.670176 399 consensus_queue.cc:1035] T 855add92bb67414cb778f98cd6c17dd8 P 62c4c4f74c3744ddbc461ef9b6acfad4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e1cf8eab0004fa8a9101887482ef407" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 34479 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:11:27.670349 392 consensus_queue.cc:1035] T 855add92bb67414cb778f98cd6c17dd8 P 62c4c4f74c3744ddbc461ef9b6acfad4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4a80a1ab47ee4291bc69d1719068abed" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 37143 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:11:27.675169 409 mvcc.cc:205] Tried to move back new op lower bound from 7315929037490425856 to 7315929037437435904. Current Snapshot: MvccSnapshot[applied={T|T < 7315929037490425856}]
I20260807 15:11:27.675804 410 mvcc.cc:205] Tried to move back new op lower bound from 7315929037490425856 to 7315929037437435904. Current Snapshot: MvccSnapshot[applied={T|T < 7315929037490425856 or (T in {7315929037490425856})}]
I20260807 15:11:27.674916 408 mvcc.cc:205] Tried to move back new op lower bound from 7315929037490425856 to 7315929037437435904. Current Snapshot: MvccSnapshot[applied={T|T < 7315929037490425856}]
I20260807 15:11:27.682030 32103 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43848:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 15:11:27.682271 32103 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43848:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 15:11:27.683579 32103 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b676891f78094012accc7df16ace058c: Sending DeleteTablet for 3 replicas of tablet 855add92bb67414cb778f98cd6c17dd8
I20260807 15:11:27.684450 32710 tablet_service.cc:1514] Processing DeleteTablet for tablet 855add92bb67414cb778f98cd6c17dd8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:11:27 UTC) from {username='kudu', principal='kudu/127.29.194.62@KRBTEST.COM'} at 127.0.0.1:34038
I20260807 15:11:27.684525 32499 tablet_service.cc:1514] Processing DeleteTablet for tablet 855add92bb67414cb778f98cd6c17dd8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:11:27 UTC) from {username='kudu', principal='kudu/127.29.194.62@KRBTEST.COM'} at 127.0.0.1:42626
I20260807 15:11:27.684896 32297 tablet_service.cc:1514] Processing DeleteTablet for tablet 855add92bb67414cb778f98cd6c17dd8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:11:27 UTC) from {username='kudu', principal='kudu/127.29.194.62@KRBTEST.COM'} at 127.0.0.1:34140
I20260807 15:11:27.685243 423 tablet_replica.cc:330] T 855add92bb67414cb778f98cd6c17dd8 P 4a80a1ab47ee4291bc69d1719068abed: stopping tablet replica
I20260807 15:11:27.685271 424 tablet_replica.cc:330] T 855add92bb67414cb778f98cd6c17dd8 P 4e1cf8eab0004fa8a9101887482ef407: stopping tablet replica
I20260807 15:11:27.685237 32103 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43848:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 15:11:27.685421 424 raft_consensus.cc:2229] T 855add92bb67414cb778f98cd6c17dd8 P 4e1cf8eab0004fa8a9101887482ef407 [term 1 FOLLOWER]: Raft consensus shutting down.
W20260807 15:11:27.685528 32103 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 15:11:27.685631 424 raft_consensus.cc:2258] T 855add92bb67414cb778f98cd6c17dd8 P 4e1cf8eab0004fa8a9101887482ef407 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:11:27.685849 425 tablet_replica.cc:330] T 855add92bb67414cb778f98cd6c17dd8 P 62c4c4f74c3744ddbc461ef9b6acfad4: stopping tablet replica
I20260807 15:11:27.686069 425 raft_consensus.cc:2229] T 855add92bb67414cb778f98cd6c17dd8 P 62c4c4f74c3744ddbc461ef9b6acfad4 [term 1 LEADER]: Raft consensus shutting down.
I20260807 15:11:27.686270 425 raft_consensus.cc:2258] T 855add92bb67414cb778f98cd6c17dd8 P 62c4c4f74c3744ddbc461ef9b6acfad4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:11:27.686332 424 ts_tablet_manager.cc:1905] T 855add92bb67414cb778f98cd6c17dd8 P 4e1cf8eab0004fa8a9101887482ef407: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:11:27.687884 32710 tablet_service.cc:1467] Processing CreateTablet for tablet 386d5751599d43fe95c17a123a960efe (DEFAULT_TABLE table=test-table [id=1bd1c73d1bed4b36a332fa0172946d1b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:11:27.688073 32710 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 386d5751599d43fe95c17a123a960efe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:11:27.688409 32499 tablet_service.cc:1467] Processing CreateTablet for tablet 386d5751599d43fe95c17a123a960efe (DEFAULT_TABLE table=test-table [id=1bd1c73d1bed4b36a332fa0172946d1b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:11:27.688611 32499 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 386d5751599d43fe95c17a123a960efe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:11:27.689073 424 ts_tablet_manager.cc:1918] T 855add92bb67414cb778f98cd6c17dd8 P 4e1cf8eab0004fa8a9101887482ef407: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 15:11:27.689240 32297 tablet_service.cc:1467] Processing CreateTablet for tablet 386d5751599d43fe95c17a123a960efe (DEFAULT_TABLE table=test-table [id=1bd1c73d1bed4b36a332fa0172946d1b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:11:27.689390 32297 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 386d5751599d43fe95c17a123a960efe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:11:27.690199 385 tablet_bootstrap.cc:492] T 386d5751599d43fe95c17a123a960efe P 4e1cf8eab0004fa8a9101887482ef407: Bootstrap starting.
I20260807 15:11:27.690423 423 raft_consensus.cc:2229] T 855add92bb67414cb778f98cd6c17dd8 P 4a80a1ab47ee4291bc69d1719068abed [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 15:11:27.690608 423 raft_consensus.cc:2258] T 855add92bb67414cb778f98cd6c17dd8 P 4a80a1ab47ee4291bc69d1719068abed [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:11:27.690801 385 tablet_bootstrap.cc:654] T 386d5751599d43fe95c17a123a960efe P 4e1cf8eab0004fa8a9101887482ef407: Neither blocks nor log segments found. Creating new log.
I20260807 15:11:27.690984 383 tablet_bootstrap.cc:492] T 386d5751599d43fe95c17a123a960efe P 62c4c4f74c3744ddbc461ef9b6acfad4: Bootstrap starting.
I20260807 15:11:27.691172 425 ts_tablet_manager.cc:1905] T 855add92bb67414cb778f98cd6c17dd8 P 62c4c4f74c3744ddbc461ef9b6acfad4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:11:27.691395 386 tablet_bootstrap.cc:492] T 386d5751599d43fe95c17a123a960efe P 4a80a1ab47ee4291bc69d1719068abed: Bootstrap starting.
I20260807 15:11:27.691469 383 tablet_bootstrap.cc:654] T 386d5751599d43fe95c17a123a960efe P 62c4c4f74c3744ddbc461ef9b6acfad4: Neither blocks nor log segments found. Creating new log.
I20260807 15:11:27.692148 386 tablet_bootstrap.cc:654] T 386d5751599d43fe95c17a123a960efe P 4a80a1ab47ee4291bc69d1719068abed: Neither blocks nor log segments found. Creating new log.
I20260807 15:11:27.692432 423 ts_tablet_manager.cc:1905] T 855add92bb67414cb778f98cd6c17dd8 P 4a80a1ab47ee4291bc69d1719068abed: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:11:27.692613 424 log.cc:1196] T 855add92bb67414cb778f98cd6c17dd8 P 4e1cf8eab0004fa8a9101887482ef407: Deleting WAL directory at /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115480851507-30472-0/minicluster-data/ts-2/wal/wals/855add92bb67414cb778f98cd6c17dd8
I20260807 15:11:27.693012 424 ts_tablet_manager.cc:1939] T 855add92bb67414cb778f98cd6c17dd8 P 4e1cf8eab0004fa8a9101887482ef407: Deleting consensus metadata
I20260807 15:11:27.693732 32082 catalog_manager.cc:4841] TS 4e1cf8eab0004fa8a9101887482ef407 (127.29.194.3:34479): tablet 855add92bb67414cb778f98cd6c17dd8 (table test-table [id=47bc612cd8f942669098b92857c70198]) successfully deleted
I20260807 15:11:27.693732 425 ts_tablet_manager.cc:1918] T 855add92bb67414cb778f98cd6c17dd8 P 62c4c4f74c3744ddbc461ef9b6acfad4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 15:11:27.694150 425 log.cc:1196] T 855add92bb67414cb778f98cd6c17dd8 P 62c4c4f74c3744ddbc461ef9b6acfad4: Deleting WAL directory at /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115480851507-30472-0/minicluster-data/ts-0/wal/wals/855add92bb67414cb778f98cd6c17dd8
I20260807 15:11:27.694561 425 ts_tablet_manager.cc:1939] T 855add92bb67414cb778f98cd6c17dd8 P 62c4c4f74c3744ddbc461ef9b6acfad4: Deleting consensus metadata
I20260807 15:11:27.694574 385 tablet_bootstrap.cc:492] T 386d5751599d43fe95c17a123a960efe P 4e1cf8eab0004fa8a9101887482ef407: No bootstrap required, opened a new log
I20260807 15:11:27.694618 385 ts_tablet_manager.cc:1397] T 386d5751599d43fe95c17a123a960efe P 4e1cf8eab0004fa8a9101887482ef407: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.001s
I20260807 15:11:27.694842 385 raft_consensus.cc:348] T 386d5751599d43fe95c17a123a960efe P 4e1cf8eab0004fa8a9101887482ef407 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62c4c4f74c3744ddbc461ef9b6acfad4" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 35187 } } peers { permanent_uuid: "4a80a1ab47ee4291bc69d1719068abed" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 37143 } } peers { permanent_uuid: "4e1cf8eab0004fa8a9101887482ef407" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 34479 } }
I20260807 15:11:27.694923 385 raft_consensus.cc:374] T 386d5751599d43fe95c17a123a960efe P 4e1cf8eab0004fa8a9101887482ef407 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:11:27.694912 423 ts_tablet_manager.cc:1918] T 855add92bb67414cb778f98cd6c17dd8 P 4a80a1ab47ee4291bc69d1719068abed: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 15:11:27.694942 385 raft_consensus.cc:729] T 386d5751599d43fe95c17a123a960efe P 4e1cf8eab0004fa8a9101887482ef407 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e1cf8eab0004fa8a9101887482ef407, State: Initialized, Role: FOLLOWER
I20260807 15:11:27.694994 423 log.cc:1196] T 855add92bb67414cb778f98cd6c17dd8 P 4a80a1ab47ee4291bc69d1719068abed: Deleting WAL directory at /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115480851507-30472-0/minicluster-data/ts-1/wal/wals/855add92bb67414cb778f98cd6c17dd8
I20260807 15:11:27.694993 385 consensus_queue.cc:260] T 386d5751599d43fe95c17a123a960efe P 4e1cf8eab0004fa8a9101887482ef407 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62c4c4f74c3744ddbc461ef9b6acfad4" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 35187 } } peers { permanent_uuid: "4a80a1ab47ee4291bc69d1719068abed" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 37143 } } peers { permanent_uuid: "4e1cf8eab0004fa8a9101887482ef407" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 34479 } }
I20260807 15:11:27.695091 385 ts_tablet_manager.cc:1428] T 386d5751599d43fe95c17a123a960efe P 4e1cf8eab0004fa8a9101887482ef407: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260807 15:11:27.695086 32081 catalog_manager.cc:4841] TS 62c4c4f74c3744ddbc461ef9b6acfad4 (127.29.194.1:35187): tablet 855add92bb67414cb778f98cd6c17dd8 (table test-table [id=47bc612cd8f942669098b92857c70198]) successfully deleted
I20260807 15:11:27.695238 423 ts_tablet_manager.cc:1939] T 855add92bb67414cb778f98cd6c17dd8 P 4a80a1ab47ee4291bc69d1719068abed: Deleting consensus metadata
I20260807 15:11:27.695564 386 tablet_bootstrap.cc:492] T 386d5751599d43fe95c17a123a960efe P 4a80a1ab47ee4291bc69d1719068abed: No bootstrap required, opened a new log
I20260807 15:11:27.695617 386 ts_tablet_manager.cc:1397] T 386d5751599d43fe95c17a123a960efe P 4a80a1ab47ee4291bc69d1719068abed: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.001s
I20260807 15:11:27.695807 32081 catalog_manager.cc:4841] TS 4a80a1ab47ee4291bc69d1719068abed (127.29.194.2:37143): tablet 855add92bb67414cb778f98cd6c17dd8 (table test-table [id=47bc612cd8f942669098b92857c70198]) successfully deleted
I20260807 15:11:27.695811 386 raft_consensus.cc:348] T 386d5751599d43fe95c17a123a960efe P 4a80a1ab47ee4291bc69d1719068abed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62c4c4f74c3744ddbc461ef9b6acfad4" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 35187 } } peers { permanent_uuid: "4a80a1ab47ee4291bc69d1719068abed" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 37143 } } peers { permanent_uuid: "4e1cf8eab0004fa8a9101887482ef407" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 34479 } }
I20260807 15:11:27.695885 386 raft_consensus.cc:374] T 386d5751599d43fe95c17a123a960efe P 4a80a1ab47ee4291bc69d1719068abed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:11:27.695904 386 raft_consensus.cc:729] T 386d5751599d43fe95c17a123a960efe P 4a80a1ab47ee4291bc69d1719068abed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4a80a1ab47ee4291bc69d1719068abed, State: Initialized, Role: FOLLOWER
I20260807 15:11:27.695951 386 consensus_queue.cc:260] T 386d5751599d43fe95c17a123a960efe P 4a80a1ab47ee4291bc69d1719068abed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62c4c4f74c3744ddbc461ef9b6acfad4" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 35187 } } peers { permanent_uuid: "4a80a1ab47ee4291bc69d1719068abed" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 37143 } } peers { permanent_uuid: "4e1cf8eab0004fa8a9101887482ef407" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 34479 } }
I20260807 15:11:27.696039 386 ts_tablet_manager.cc:1428] T 386d5751599d43fe95c17a123a960efe P 4a80a1ab47ee4291bc69d1719068abed: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260807 15:11:27.697854 383 tablet_bootstrap.cc:492] T 386d5751599d43fe95c17a123a960efe P 62c4c4f74c3744ddbc461ef9b6acfad4: No bootstrap required, opened a new log
I20260807 15:11:27.697911 383 ts_tablet_manager.cc:1397] T 386d5751599d43fe95c17a123a960efe P 62c4c4f74c3744ddbc461ef9b6acfad4: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.000s
I20260807 15:11:27.698063 383 raft_consensus.cc:348] T 386d5751599d43fe95c17a123a960efe P 62c4c4f74c3744ddbc461ef9b6acfad4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62c4c4f74c3744ddbc461ef9b6acfad4" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 35187 } } peers { permanent_uuid: "4a80a1ab47ee4291bc69d1719068abed" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 37143 } } peers { permanent_uuid: "4e1cf8eab0004fa8a9101887482ef407" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 34479 } }
I20260807 15:11:27.698127 383 raft_consensus.cc:374] T 386d5751599d43fe95c17a123a960efe P 62c4c4f74c3744ddbc461ef9b6acfad4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:11:27.698144 383 raft_consensus.cc:729] T 386d5751599d43fe95c17a123a960efe P 62c4c4f74c3744ddbc461ef9b6acfad4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 62c4c4f74c3744ddbc461ef9b6acfad4, State: Initialized, Role: FOLLOWER
I20260807 15:11:27.698190 383 consensus_queue.cc:260] T 386d5751599d43fe95c17a123a960efe P 62c4c4f74c3744ddbc461ef9b6acfad4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62c4c4f74c3744ddbc461ef9b6acfad4" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 35187 } } peers { permanent_uuid: "4a80a1ab47ee4291bc69d1719068abed" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 37143 } } peers { permanent_uuid: "4e1cf8eab0004fa8a9101887482ef407" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 34479 } }
I20260807 15:11:27.698298 383 ts_tablet_manager.cc:1428] T 386d5751599d43fe95c17a123a960efe P 62c4c4f74c3744ddbc461ef9b6acfad4: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
W20260807 15:11:27.737843 348 tablet.cc:2369] T 386d5751599d43fe95c17a123a960efe P 4e1cf8eab0004fa8a9101887482ef407: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 15:11:27.780455 32599 tablet.cc:2369] T 386d5751599d43fe95c17a123a960efe P 4a80a1ab47ee4291bc69d1719068abed: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 15:11:27.815828 417 raft_consensus.cc:482] T 386d5751599d43fe95c17a123a960efe P 62c4c4f74c3744ddbc461ef9b6acfad4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 15:11:27.815963 417 raft_consensus.cc:504] T 386d5751599d43fe95c17a123a960efe P 62c4c4f74c3744ddbc461ef9b6acfad4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62c4c4f74c3744ddbc461ef9b6acfad4" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 35187 } } peers { permanent_uuid: "4a80a1ab47ee4291bc69d1719068abed" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 37143 } } peers { permanent_uuid: "4e1cf8eab0004fa8a9101887482ef407" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 34479 } }
I20260807 15:11:27.816147 417 leader_election.cc:288] T 386d5751599d43fe95c17a123a960efe P 62c4c4f74c3744ddbc461ef9b6acfad4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4a80a1ab47ee4291bc69d1719068abed (127.29.194.2:37143), 4e1cf8eab0004fa8a9101887482ef407 (127.29.194.3:34479)
I20260807 15:11:27.816478 32527 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "386d5751599d43fe95c17a123a960efe" candidate_uuid: "62c4c4f74c3744ddbc461ef9b6acfad4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4a80a1ab47ee4291bc69d1719068abed" is_pre_election: true
I20260807 15:11:27.816483 32743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "386d5751599d43fe95c17a123a960efe" candidate_uuid: "62c4c4f74c3744ddbc461ef9b6acfad4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e1cf8eab0004fa8a9101887482ef407" is_pre_election: true
I20260807 15:11:27.816592 32743 raft_consensus.cc:2454] T 386d5751599d43fe95c17a123a960efe P 4e1cf8eab0004fa8a9101887482ef407 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 62c4c4f74c3744ddbc461ef9b6acfad4 in term 0.
I20260807 15:11:27.816592 32527 raft_consensus.cc:2454] T 386d5751599d43fe95c17a123a960efe P 4a80a1ab47ee4291bc69d1719068abed [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 62c4c4f74c3744ddbc461ef9b6acfad4 in term 0.
I20260807 15:11:27.816756 32233 leader_election.cc:302] T 386d5751599d43fe95c17a123a960efe P 62c4c4f74c3744ddbc461ef9b6acfad4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e1cf8eab0004fa8a9101887482ef407, 62c4c4f74c3744ddbc461ef9b6acfad4; no voters:
I20260807 15:11:27.816900 417 raft_consensus.cc:2781] T 386d5751599d43fe95c17a123a960efe P 62c4c4f74c3744ddbc461ef9b6acfad4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 15:11:27.816938 417 raft_consensus.cc:482] T 386d5751599d43fe95c17a123a960efe P 62c4c4f74c3744ddbc461ef9b6acfad4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 15:11:27.816957 417 raft_consensus.cc:3037] T 386d5751599d43fe95c17a123a960efe P 62c4c4f74c3744ddbc461ef9b6acfad4 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:11:27.817963 417 raft_consensus.cc:504] T 386d5751599d43fe95c17a123a960efe P 62c4c4f74c3744ddbc461ef9b6acfad4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62c4c4f74c3744ddbc461ef9b6acfad4" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 35187 } } peers { permanent_uuid: "4a80a1ab47ee4291bc69d1719068abed" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 37143 } } peers { permanent_uuid: "4e1cf8eab0004fa8a9101887482ef407" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 34479 } }
I20260807 15:11:27.818123 417 leader_election.cc:288] T 386d5751599d43fe95c17a123a960efe P 62c4c4f74c3744ddbc461ef9b6acfad4 [CANDIDATE]: Term 1 election: Requested vote from peers 4a80a1ab47ee4291bc69d1719068abed (127.29.194.2:37143), 4e1cf8eab0004fa8a9101887482ef407 (127.29.194.3:34479)
I20260807 15:11:27.818356 32743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "386d5751599d43fe95c17a123a960efe" candidate_uuid: "62c4c4f74c3744ddbc461ef9b6acfad4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e1cf8eab0004fa8a9101887482ef407"
I20260807 15:11:27.818389 32527 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "386d5751599d43fe95c17a123a960efe" candidate_uuid: "62c4c4f74c3744ddbc461ef9b6acfad4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4a80a1ab47ee4291bc69d1719068abed"
I20260807 15:11:27.818440 32743 raft_consensus.cc:3037] T 386d5751599d43fe95c17a123a960efe P 4e1cf8eab0004fa8a9101887482ef407 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:11:27.818472 32527 raft_consensus.cc:3037] T 386d5751599d43fe95c17a123a960efe P 4a80a1ab47ee4291bc69d1719068abed [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:11:27.819226 32743 raft_consensus.cc:2454] T 386d5751599d43fe95c17a123a960efe P 4e1cf8eab0004fa8a9101887482ef407 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 62c4c4f74c3744ddbc461ef9b6acfad4 in term 1.
I20260807 15:11:27.819387 32527 raft_consensus.cc:2454] T 386d5751599d43fe95c17a123a960efe P 4a80a1ab47ee4291bc69d1719068abed [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 62c4c4f74c3744ddbc461ef9b6acfad4 in term 1.
I20260807 15:11:27.819460 32233 leader_election.cc:302] T 386d5751599d43fe95c17a123a960efe P 62c4c4f74c3744ddbc461ef9b6acfad4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e1cf8eab0004fa8a9101887482ef407, 62c4c4f74c3744ddbc461ef9b6acfad4; no voters:
I20260807 15:11:27.819660 417 raft_consensus.cc:2781] T 386d5751599d43fe95c17a123a960efe P 62c4c4f74c3744ddbc461ef9b6acfad4 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:11:27.819707 417 raft_consensus.cc:686] T 386d5751599d43fe95c17a123a960efe P 62c4c4f74c3744ddbc461ef9b6acfad4 [term 1 LEADER]: Becoming Leader. State: Replica: 62c4c4f74c3744ddbc461ef9b6acfad4, State: Running, Role: LEADER
I20260807 15:11:27.819746 417 consensus_queue.cc:237] T 386d5751599d43fe95c17a123a960efe P 62c4c4f74c3744ddbc461ef9b6acfad4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62c4c4f74c3744ddbc461ef9b6acfad4" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 35187 } } peers { permanent_uuid: "4a80a1ab47ee4291bc69d1719068abed" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 37143 } } peers { permanent_uuid: "4e1cf8eab0004fa8a9101887482ef407" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 34479 } }
I20260807 15:11:27.820305 32105 catalog_manager.cc:5495] T 386d5751599d43fe95c17a123a960efe P 62c4c4f74c3744ddbc461ef9b6acfad4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 62c4c4f74c3744ddbc461ef9b6acfad4 (127.29.194.1). New cstate: current_term: 1 leader_uuid: "62c4c4f74c3744ddbc461ef9b6acfad4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62c4c4f74c3744ddbc461ef9b6acfad4" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 35187 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4a80a1ab47ee4291bc69d1719068abed" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 37143 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e1cf8eab0004fa8a9101887482ef407" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 34479 } health_report { overall_health: UNKNOWN } } }
Aug 07 15:11:27 dist-test-slave-42z9 krb5kdc[32035](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115486, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.194.62@KRBTEST.COM for kudu/127.29.194.62@KRBTEST.COM
I20260807 15:11:27.882653 440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:27.874359 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:45747 (local address 127.0.0.1:43854)
0807 15:11:27.874560 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0807 15:11:27.874570 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 15:11:27.874632 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:11:27.874778 (+ 146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:11:27.874779 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:11:27.875258 (+ 479us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:11:27.875364 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:27.875372 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:27.876456 (+ 1084us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:27.876458 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:27.877034 (+ 576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:27.877036 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:27.877544 (+ 508us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:27.877546 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:27.877611 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:27.878158 (+ 547us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:11:27.878162 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:11:27.879607 (+ 1445us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:11:27.881537 (+ 1930us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:11:27.881546 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:11:27.881549 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:11:27.881758 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:11:27.881974 (+ 216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:11:27.881975 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:11:27.881975 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:11:27.882042 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:11:27.882322 (+ 280us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:11:27.882329 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:11:27.882379 (+ 50us) client_negotiation.cc:769] Sending connection context
0807 15:11:27.882444 (+ 65us) client_negotiation.cc:240] Negotiation successful
0807 15:11:27.882575 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":114,"threads_started":1}
I20260807 15:11:27.882763 32170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:27.874444 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43854 (local address 127.29.194.62:45747)
0807 15:11:27.874524 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:27.874527 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:27.874630 (+ 103us) server_negotiation.cc:407] Connection header received
0807 15:11:27.874680 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:27.874681 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:27.874692 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:27.874773 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:11:27.875524 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:27.876323 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:27.877125 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:27.877417 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:27.877506 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:27.879837 (+ 2331us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:11:27.879838 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:11:27.879839 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:11:27.879847 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:11:27.881387 (+ 1540us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:27.881830 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:27.881831 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:27.881832 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:27.881896 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:27.882097 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:27.882097 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:27.882098 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:27.882240 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:11:27.882304 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0807 15:11:27.882476 (+ 172us) server_negotiation.cc:299] Negotiation successful
0807 15:11:27.882646 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":85,"server-negotiator.queue_time_us":53}
I20260807 15:11:27.885195 32105 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.29.194.62@KRBTEST.COM'} at 127.0.0.1:43854:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260807 15:11:27.885586 32105 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 15:11:27.887161 32710 tablet_service.cc:1467] Processing CreateTablet for tablet 1314a2e9bc9f4ffe84f1e1b933899c2e (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=c472df4971eb49e2beada94eed781abf]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 15:11:27.887212 32499 tablet_service.cc:1467] Processing CreateTablet for tablet 1314a2e9bc9f4ffe84f1e1b933899c2e (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=c472df4971eb49e2beada94eed781abf]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 15:11:27.887394 32499 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1314a2e9bc9f4ffe84f1e1b933899c2e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:11:27.887395 32710 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1314a2e9bc9f4ffe84f1e1b933899c2e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:11:27.888882 32297 tablet_service.cc:1467] Processing CreateTablet for tablet 1314a2e9bc9f4ffe84f1e1b933899c2e (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=c472df4971eb49e2beada94eed781abf]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 15:11:27.889021 32297 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1314a2e9bc9f4ffe84f1e1b933899c2e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:11:27.910053 383 tablet_bootstrap.cc:492] T 1314a2e9bc9f4ffe84f1e1b933899c2e P 62c4c4f74c3744ddbc461ef9b6acfad4: Bootstrap starting.
I20260807 15:11:27.910076 385 tablet_bootstrap.cc:492] T 1314a2e9bc9f4ffe84f1e1b933899c2e P 4e1cf8eab0004fa8a9101887482ef407: Bootstrap starting.
I20260807 15:11:27.910208 386 tablet_bootstrap.cc:492] T 1314a2e9bc9f4ffe84f1e1b933899c2e P 4a80a1ab47ee4291bc69d1719068abed: Bootstrap starting.
I20260807 15:11:27.910537 383 tablet_bootstrap.cc:654] T 1314a2e9bc9f4ffe84f1e1b933899c2e P 62c4c4f74c3744ddbc461ef9b6acfad4: Neither blocks nor log segments found. Creating new log.
I20260807 15:11:27.910686 386 tablet_bootstrap.cc:654] T 1314a2e9bc9f4ffe84f1e1b933899c2e P 4a80a1ab47ee4291bc69d1719068abed: Neither blocks nor log segments found. Creating new log.
I20260807 15:11:27.910686 385 tablet_bootstrap.cc:654] T 1314a2e9bc9f4ffe84f1e1b933899c2e P 4e1cf8eab0004fa8a9101887482ef407: Neither blocks nor log segments found. Creating new log.
I20260807 15:11:27.911275 383 tablet_bootstrap.cc:492] T 1314a2e9bc9f4ffe84f1e1b933899c2e P 62c4c4f74c3744ddbc461ef9b6acfad4: No bootstrap required, opened a new log
I20260807 15:11:27.911337 383 ts_tablet_manager.cc:1397] T 1314a2e9bc9f4ffe84f1e1b933899c2e P 62c4c4f74c3744ddbc461ef9b6acfad4: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260807 15:11:27.911360 386 tablet_bootstrap.cc:492] T 1314a2e9bc9f4ffe84f1e1b933899c2e P 4a80a1ab47ee4291bc69d1719068abed: No bootstrap required, opened a new log
I20260807 15:11:27.911406 386 ts_tablet_manager.cc:1397] T 1314a2e9bc9f4ffe84f1e1b933899c2e P 4a80a1ab47ee4291bc69d1719068abed: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260807 15:11:27.911435 385 tablet_bootstrap.cc:492] T 1314a2e9bc9f4ffe84f1e1b933899c2e P 4e1cf8eab0004fa8a9101887482ef407: No bootstrap required, opened a new log
I20260807 15:11:27.911486 385 ts_tablet_manager.cc:1397] T 1314a2e9bc9f4ffe84f1e1b933899c2e P 4e1cf8eab0004fa8a9101887482ef407: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260807 15:11:27.911527 383 raft_consensus.cc:348] T 1314a2e9bc9f4ffe84f1e1b933899c2e P 62c4c4f74c3744ddbc461ef9b6acfad4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62c4c4f74c3744ddbc461ef9b6acfad4" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 35187 } } peers { permanent_uuid: "4a80a1ab47ee4291bc69d1719068abed" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 37143 } } peers { permanent_uuid: "4e1cf8eab0004fa8a9101887482ef407" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 34479 } }
I20260807 15:11:27.911614 383 raft_consensus.cc:374] T 1314a2e9bc9f4ffe84f1e1b933899c2e P 62c4c4f74c3744ddbc461ef9b6acfad4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:11:27.911634 383 raft_consensus.cc:729] T 1314a2e9bc9f4ffe84f1e1b933899c2e P 62c4c4f74c3744ddbc461ef9b6acfad4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 62c4c4f74c3744ddbc461ef9b6acfad4, State: Initialized, Role: FOLLOWER
I20260807 15:11:27.911614 386 raft_consensus.cc:348] T 1314a2e9bc9f4ffe84f1e1b933899c2e P 4a80a1ab47ee4291bc69d1719068abed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62c4c4f74c3744ddbc461ef9b6acfad4" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 35187 } } peers { permanent_uuid: "4a80a1ab47ee4291bc69d1719068abed" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 37143 } } peers { permanent_uuid: "4e1cf8eab0004fa8a9101887482ef407" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 34479 } }
I20260807 15:11:27.911698 386 raft_consensus.cc:374] T 1314a2e9bc9f4ffe84f1e1b933899c2e P 4a80a1ab47ee4291bc69d1719068abed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:11:27.911681 383 consensus_queue.cc:260] T 1314a2e9bc9f4ffe84f1e1b933899c2e P 62c4c4f74c3744ddbc461ef9b6acfad4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62c4c4f74c3744ddbc461ef9b6acfad4" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 35187 } } peers { permanent_uuid: "4a80a1ab47ee4291bc69d1719068abed" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 37143 } } peers { permanent_uuid: "4e1cf8eab0004fa8a9101887482ef407" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 34479 } }
I20260807 15:11:27.911651 385 raft_consensus.cc:348] T 1314a2e9bc9f4ffe84f1e1b933899c2e P 4e1cf8eab0004fa8a9101887482ef407 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62c4c4f74c3744ddbc461ef9b6acfad4" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 35187 } } peers { permanent_uuid: "4a80a1ab47ee4291bc69d1719068abed" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 37143 } } peers { permanent_uuid: "4e1cf8eab0004fa8a9101887482ef407" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 34479 } }
I20260807 15:11:27.911720 386 raft_consensus.cc:729] T 1314a2e9bc9f4ffe84f1e1b933899c2e P 4a80a1ab47ee4291bc69d1719068abed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4a80a1ab47ee4291bc69d1719068abed, State: Initialized, Role: FOLLOWER
I20260807 15:11:27.911723 385 raft_consensus.cc:374] T 1314a2e9bc9f4ffe84f1e1b933899c2e P 4e1cf8eab0004fa8a9101887482ef407 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:11:27.911743 385 raft_consensus.cc:729] T 1314a2e9bc9f4ffe84f1e1b933899c2e P 4e1cf8eab0004fa8a9101887482ef407 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e1cf8eab0004fa8a9101887482ef407, State: Initialized, Role: FOLLOWER
I20260807 15:11:27.911770 383 ts_tablet_manager.cc:1428] T 1314a2e9bc9f4ffe84f1e1b933899c2e P 62c4c4f74c3744ddbc461ef9b6acfad4: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260807 15:11:27.911768 386 consensus_queue.cc:260] T 1314a2e9bc9f4ffe84f1e1b933899c2e P 4a80a1ab47ee4291bc69d1719068abed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62c4c4f74c3744ddbc461ef9b6acfad4" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 35187 } } peers { permanent_uuid: "4a80a1ab47ee4291bc69d1719068abed" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 37143 } } peers { permanent_uuid: "4e1cf8eab0004fa8a9101887482ef407" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 34479 } }
I20260807 15:11:27.911782 385 consensus_queue.cc:260] T 1314a2e9bc9f4ffe84f1e1b933899c2e P 4e1cf8eab0004fa8a9101887482ef407 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62c4c4f74c3744ddbc461ef9b6acfad4" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 35187 } } peers { permanent_uuid: "4a80a1ab47ee4291bc69d1719068abed" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 37143 } } peers { permanent_uuid: "4e1cf8eab0004fa8a9101887482ef407" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 34479 } }
I20260807 15:11:27.911859 385 ts_tablet_manager.cc:1428] T 1314a2e9bc9f4ffe84f1e1b933899c2e P 4e1cf8eab0004fa8a9101887482ef407: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260807 15:11:27.911857 386 ts_tablet_manager.cc:1428] T 1314a2e9bc9f4ffe84f1e1b933899c2e P 4a80a1ab47ee4291bc69d1719068abed: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260807 15:11:27.911887 392 tablet_replica.cc:439] T 1314a2e9bc9f4ffe84f1e1b933899c2e P 62c4c4f74c3744ddbc461ef9b6acfad4: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62c4c4f74c3744ddbc461ef9b6acfad4" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 35187 } } peers { permanent_uuid: "4a80a1ab47ee4291bc69d1719068abed" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 37143 } } peers { permanent_uuid: "4e1cf8eab0004fa8a9101887482ef407" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 34479 } } }
I20260807 15:11:27.911962 392 tablet_replica.cc:442] T 1314a2e9bc9f4ffe84f1e1b933899c2e P 62c4c4f74c3744ddbc461ef9b6acfad4: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 15:11:27.912014 393 tablet_replica.cc:439] T 1314a2e9bc9f4ffe84f1e1b933899c2e P 4a80a1ab47ee4291bc69d1719068abed: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62c4c4f74c3744ddbc461ef9b6acfad4" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 35187 } } peers { permanent_uuid: "4a80a1ab47ee4291bc69d1719068abed" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 37143 } } peers { permanent_uuid: "4e1cf8eab0004fa8a9101887482ef407" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 34479 } } }
I20260807 15:11:27.912029 396 tablet_replica.cc:439] T 1314a2e9bc9f4ffe84f1e1b933899c2e P 4e1cf8eab0004fa8a9101887482ef407: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62c4c4f74c3744ddbc461ef9b6acfad4" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 35187 } } peers { permanent_uuid: "4a80a1ab47ee4291bc69d1719068abed" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 37143 } } peers { permanent_uuid: "4e1cf8eab0004fa8a9101887482ef407" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 34479 } } }
I20260807 15:11:27.912137 396 tablet_replica.cc:442] T 1314a2e9bc9f4ffe84f1e1b933899c2e P 4e1cf8eab0004fa8a9101887482ef407: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 15:11:27.912137 393 tablet_replica.cc:442] T 1314a2e9bc9f4ffe84f1e1b933899c2e P 4a80a1ab47ee4291bc69d1719068abed: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 15:11:28.152063 393 raft_consensus.cc:482] T 1314a2e9bc9f4ffe84f1e1b933899c2e P 4a80a1ab47ee4291bc69d1719068abed [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 15:11:28.152194 393 raft_consensus.cc:504] T 1314a2e9bc9f4ffe84f1e1b933899c2e P 4a80a1ab47ee4291bc69d1719068abed [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62c4c4f74c3744ddbc461ef9b6acfad4" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 35187 } } peers { permanent_uuid: "4a80a1ab47ee4291bc69d1719068abed" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 37143 } } peers { permanent_uuid: "4e1cf8eab0004fa8a9101887482ef407" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 34479 } }
I20260807 15:11:28.152554 393 leader_election.cc:288] T 1314a2e9bc9f4ffe84f1e1b933899c2e P 4a80a1ab47ee4291bc69d1719068abed [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 62c4c4f74c3744ddbc461ef9b6acfad4 (127.29.194.1:35187), 4e1cf8eab0004fa8a9101887482ef407 (127.29.194.3:34479)
I20260807 15:11:28.155728 445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:28.152777 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.3:34479 (local address 127.29.194.2:52805)
0807 15:11:28.152916 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0807 15:11:28.152926 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 15:11:28.152978 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:11:28.153246 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:11:28.153247 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:11:28.153249 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:11:28.153319 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:28.153322 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:28.153994 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:28.153995 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:28.154839 (+ 844us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:28.154842 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:28.155444 (+ 602us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:28.155445 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:28.155522 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:28.155526 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 15:11:28.155570 (+ 44us) client_negotiation.cc:240] Negotiation successful
0807 15:11:28.155622 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":63,"threads_started":1}
I20260807 15:11:28.155706 447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:28.153013 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.194.2:52805 (local address 127.29.194.3:34479)
0807 15:11:28.153132 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:28.153133 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:28.153140 (+ 7us) server_negotiation.cc:407] Connection header received
0807 15:11:28.153164 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:28.153165 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:28.153174 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:28.153230 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:11:28.153379 (+ 149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:28.153911 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:28.154949 (+ 1038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:28.155366 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:28.155427 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:28.155438 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0807 15:11:28.155587 (+ 149us) server_negotiation.cc:299] Negotiation successful
0807 15:11:28.155630 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":100,"thread_start_us":66,"threads_started":1}
I20260807 15:11:28.155707 382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:28.152968 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.194.2:33017 (local address 127.29.194.1:35187)
0807 15:11:28.153026 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:28.153028 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:28.153129 (+ 101us) server_negotiation.cc:407] Connection header received
0807 15:11:28.153200 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:28.153200 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:28.153212 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:28.153255 (+ 43us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:11:28.153471 (+ 216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:28.153929 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:28.154992 (+ 1063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:28.155380 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:28.155453 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:28.155462 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0807 15:11:28.155596 (+ 134us) server_negotiation.cc:299] Negotiation successful
0807 15:11:28.155636 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260807 15:11:28.155743 446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:28.152777 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.1:35187 (local address 127.29.194.2:33017)
0807 15:11:28.153050 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0807 15:11:28.153060 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 15:11:28.153136 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:11:28.153269 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:11:28.153270 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:11:28.153272 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:11:28.153379 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:28.153380 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:28.154011 (+ 631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:28.154012 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:28.154841 (+ 829us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:28.154843 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:28.155474 (+ 631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:28.155475 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:28.155537 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:28.155540 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 15:11:28.155586 (+ 46us) client_negotiation.cc:240] Negotiation successful
0807 15:11:28.155630 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":258,"mutex_wait_us":17,"thread_start_us":48,"threads_started":1}
I20260807 15:11:28.156100 32743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1314a2e9bc9f4ffe84f1e1b933899c2e" candidate_uuid: "4a80a1ab47ee4291bc69d1719068abed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e1cf8eab0004fa8a9101887482ef407" is_pre_election: true
I20260807 15:11:28.156201 32743 raft_consensus.cc:2454] T 1314a2e9bc9f4ffe84f1e1b933899c2e P 4e1cf8eab0004fa8a9101887482ef407 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4a80a1ab47ee4291bc69d1719068abed in term 0.
I20260807 15:11:28.156136 32331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1314a2e9bc9f4ffe84f1e1b933899c2e" candidate_uuid: "4a80a1ab47ee4291bc69d1719068abed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "62c4c4f74c3744ddbc461ef9b6acfad4" is_pre_election: true
I20260807 15:11:28.156322 32331 raft_consensus.cc:2454] T 1314a2e9bc9f4ffe84f1e1b933899c2e P 62c4c4f74c3744ddbc461ef9b6acfad4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4a80a1ab47ee4291bc69d1719068abed in term 0.
I20260807 15:11:28.156402 32434 leader_election.cc:302] T 1314a2e9bc9f4ffe84f1e1b933899c2e P 4a80a1ab47ee4291bc69d1719068abed [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4a80a1ab47ee4291bc69d1719068abed, 4e1cf8eab0004fa8a9101887482ef407; no voters:
I20260807 15:11:28.156551 393 raft_consensus.cc:2781] T 1314a2e9bc9f4ffe84f1e1b933899c2e P 4a80a1ab47ee4291bc69d1719068abed [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 15:11:28.156602 393 raft_consensus.cc:482] T 1314a2e9bc9f4ffe84f1e1b933899c2e P 4a80a1ab47ee4291bc69d1719068abed [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 15:11:28.156618 393 raft_consensus.cc:3037] T 1314a2e9bc9f4ffe84f1e1b933899c2e P 4a80a1ab47ee4291bc69d1719068abed [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:11:28.157342 393 raft_consensus.cc:504] T 1314a2e9bc9f4ffe84f1e1b933899c2e P 4a80a1ab47ee4291bc69d1719068abed [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62c4c4f74c3744ddbc461ef9b6acfad4" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 35187 } } peers { permanent_uuid: "4a80a1ab47ee4291bc69d1719068abed" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 37143 } } peers { permanent_uuid: "4e1cf8eab0004fa8a9101887482ef407" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 34479 } }
I20260807 15:11:28.157519 393 leader_election.cc:288] T 1314a2e9bc9f4ffe84f1e1b933899c2e P 4a80a1ab47ee4291bc69d1719068abed [CANDIDATE]: Term 1 election: Requested vote from peers 62c4c4f74c3744ddbc461ef9b6acfad4 (127.29.194.1:35187), 4e1cf8eab0004fa8a9101887482ef407 (127.29.194.3:34479)
I20260807 15:11:28.157696 32331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1314a2e9bc9f4ffe84f1e1b933899c2e" candidate_uuid: "4a80a1ab47ee4291bc69d1719068abed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "62c4c4f74c3744ddbc461ef9b6acfad4"
I20260807 15:11:28.157780 32331 raft_consensus.cc:3037] T 1314a2e9bc9f4ffe84f1e1b933899c2e P 62c4c4f74c3744ddbc461ef9b6acfad4 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:11:28.157769 32743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1314a2e9bc9f4ffe84f1e1b933899c2e" candidate_uuid: "4a80a1ab47ee4291bc69d1719068abed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e1cf8eab0004fa8a9101887482ef407"
I20260807 15:11:28.157864 32743 raft_consensus.cc:3037] T 1314a2e9bc9f4ffe84f1e1b933899c2e P 4e1cf8eab0004fa8a9101887482ef407 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:11:28.158507 32331 raft_consensus.cc:2454] T 1314a2e9bc9f4ffe84f1e1b933899c2e P 62c4c4f74c3744ddbc461ef9b6acfad4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4a80a1ab47ee4291bc69d1719068abed in term 1.
I20260807 15:11:28.158527 32743 raft_consensus.cc:2454] T 1314a2e9bc9f4ffe84f1e1b933899c2e P 4e1cf8eab0004fa8a9101887482ef407 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4a80a1ab47ee4291bc69d1719068abed in term 1.
I20260807 15:11:28.158676 32434 leader_election.cc:302] T 1314a2e9bc9f4ffe84f1e1b933899c2e P 4a80a1ab47ee4291bc69d1719068abed [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4a80a1ab47ee4291bc69d1719068abed, 4e1cf8eab0004fa8a9101887482ef407; no voters:
I20260807 15:11:28.158813 393 raft_consensus.cc:2781] T 1314a2e9bc9f4ffe84f1e1b933899c2e P 4a80a1ab47ee4291bc69d1719068abed [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:11:28.158948 393 raft_consensus.cc:686] T 1314a2e9bc9f4ffe84f1e1b933899c2e P 4a80a1ab47ee4291bc69d1719068abed [term 1 LEADER]: Becoming Leader. State: Replica: 4a80a1ab47ee4291bc69d1719068abed, State: Running, Role: LEADER
I20260807 15:11:28.159017 393 consensus_queue.cc:237] T 1314a2e9bc9f4ffe84f1e1b933899c2e P 4a80a1ab47ee4291bc69d1719068abed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62c4c4f74c3744ddbc461ef9b6acfad4" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 35187 } } peers { permanent_uuid: "4a80a1ab47ee4291bc69d1719068abed" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 37143 } } peers { permanent_uuid: "4e1cf8eab0004fa8a9101887482ef407" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 34479 } }
I20260807 15:11:28.159349 449 tablet_replica.cc:439] T 1314a2e9bc9f4ffe84f1e1b933899c2e P 4a80a1ab47ee4291bc69d1719068abed: TxnStatusTablet state changed. Reason: New leader 4a80a1ab47ee4291bc69d1719068abed. Latest consensus state: current_term: 1 leader_uuid: "4a80a1ab47ee4291bc69d1719068abed" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62c4c4f74c3744ddbc461ef9b6acfad4" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 35187 } } peers { permanent_uuid: "4a80a1ab47ee4291bc69d1719068abed" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 37143 } } peers { permanent_uuid: "4e1cf8eab0004fa8a9101887482ef407" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 34479 } } }
I20260807 15:11:28.159456 449 tablet_replica.cc:442] T 1314a2e9bc9f4ffe84f1e1b933899c2e P 4a80a1ab47ee4291bc69d1719068abed: This TxnStatusTablet replica's current role is: LEADER
I20260807 15:11:28.159694 453 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260807 15:11:28.159799 453 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260807 15:11:28.159760 32102 catalog_manager.cc:5495] T 1314a2e9bc9f4ffe84f1e1b933899c2e P 4a80a1ab47ee4291bc69d1719068abed reported cstate change: term changed from 0 to 1, leader changed from <none> to 4a80a1ab47ee4291bc69d1719068abed (127.29.194.2). New cstate: current_term: 1 leader_uuid: "4a80a1ab47ee4291bc69d1719068abed" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62c4c4f74c3744ddbc461ef9b6acfad4" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 35187 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4a80a1ab47ee4291bc69d1719068abed" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 37143 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4e1cf8eab0004fa8a9101887482ef407" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 34479 } health_report { overall_health: UNKNOWN } } }
I20260807 15:11:28.179302 462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:28.176633 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42640 (local address 127.29.194.2:37143)
0807 15:11:28.176743 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:28.176745 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:28.176754 (+ 9us) server_negotiation.cc:407] Connection header received
0807 15:11:28.176801 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:28.176802 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:28.176823 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:28.176894 (+ 71us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 15:11:28.177034 (+ 140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:28.177729 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:28.178512 (+ 783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:28.178724 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:28.178780 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:28.178966 (+ 186us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 15:11:28.179070 (+ 104us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 15:11:28.179133 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0807 15:11:28.179176 (+ 43us) server_negotiation.cc:299] Negotiation successful
0807 15:11:28.179199 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90,"thread_start_us":53,"threads_started":1}
I20260807 15:11:28.179299 440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:28.176561 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.2:37143 (local address 127.0.0.1:42640)
0807 15:11:28.176648 (+ 87us) negotiation.cc:107] Waiting for socket to connect
0807 15:11:28.176658 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 15:11:28.176699 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:11:28.176899 (+ 200us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:11:28.176900 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:11:28.176902 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 15:11:28.176969 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:28.176972 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:28.177821 (+ 849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:28.177823 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:28.178432 (+ 609us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:28.178434 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:28.178786 (+ 352us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:28.178787 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:28.178857 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:28.178878 (+ 21us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 15:11:28.179123 (+ 245us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 15:11:28.179131 (+ 8us) client_negotiation.cc:769] Sending connection context
0807 15:11:28.179174 (+ 43us) client_negotiation.cc:240] Negotiation successful
0807 15:11:28.179231 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":51}
I20260807 15:11:28.180670 32743 raft_consensus.cc:1261] T 1314a2e9bc9f4ffe84f1e1b933899c2e P 4e1cf8eab0004fa8a9101887482ef407 [term 1 FOLLOWER]: Refusing update from remote peer 4a80a1ab47ee4291bc69d1719068abed: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:11:28.180864 32331 raft_consensus.cc:1261] T 1314a2e9bc9f4ffe84f1e1b933899c2e P 62c4c4f74c3744ddbc461ef9b6acfad4 [term 1 FOLLOWER]: Refusing update from remote peer 4a80a1ab47ee4291bc69d1719068abed: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:11:28.181221 449 consensus_queue.cc:1035] T 1314a2e9bc9f4ffe84f1e1b933899c2e P 4a80a1ab47ee4291bc69d1719068abed [LEADER]: Connected to new peer: Peer: permanent_uuid: "62c4c4f74c3744ddbc461ef9b6acfad4" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 35187 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:11:28.181352 393 consensus_queue.cc:1035] T 1314a2e9bc9f4ffe84f1e1b933899c2e P 4a80a1ab47ee4291bc69d1719068abed [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e1cf8eab0004fa8a9101887482ef407" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 34479 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:11:28.181979 396 tablet_replica.cc:439] T 1314a2e9bc9f4ffe84f1e1b933899c2e P 4e1cf8eab0004fa8a9101887482ef407: TxnStatusTablet state changed. Reason: New leader 4a80a1ab47ee4291bc69d1719068abed. Latest consensus state: current_term: 1 leader_uuid: "4a80a1ab47ee4291bc69d1719068abed" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62c4c4f74c3744ddbc461ef9b6acfad4" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 35187 } } peers { permanent_uuid: "4a80a1ab47ee4291bc69d1719068abed" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 37143 } } peers { permanent_uuid: "4e1cf8eab0004fa8a9101887482ef407" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 34479 } } }
I20260807 15:11:28.182081 396 tablet_replica.cc:442] T 1314a2e9bc9f4ffe84f1e1b933899c2e P 4e1cf8eab0004fa8a9101887482ef407: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 15:11:28.182180 392 tablet_replica.cc:439] T 1314a2e9bc9f4ffe84f1e1b933899c2e P 62c4c4f74c3744ddbc461ef9b6acfad4: TxnStatusTablet state changed. Reason: New leader 4a80a1ab47ee4291bc69d1719068abed. Latest consensus state: current_term: 1 leader_uuid: "4a80a1ab47ee4291bc69d1719068abed" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62c4c4f74c3744ddbc461ef9b6acfad4" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 35187 } } peers { permanent_uuid: "4a80a1ab47ee4291bc69d1719068abed" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 37143 } } peers { permanent_uuid: "4e1cf8eab0004fa8a9101887482ef407" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 34479 } } }
I20260807 15:11:28.182256 392 tablet_replica.cc:442] T 1314a2e9bc9f4ffe84f1e1b933899c2e P 62c4c4f74c3744ddbc461ef9b6acfad4: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 15:11:28.183030 449 tablet_replica.cc:439] T 1314a2e9bc9f4ffe84f1e1b933899c2e P 4a80a1ab47ee4291bc69d1719068abed: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "4a80a1ab47ee4291bc69d1719068abed" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62c4c4f74c3744ddbc461ef9b6acfad4" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 35187 } } peers { permanent_uuid: "4a80a1ab47ee4291bc69d1719068abed" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 37143 } } peers { permanent_uuid: "4e1cf8eab0004fa8a9101887482ef407" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 34479 } } }
I20260807 15:11:28.183115 449 tablet_replica.cc:442] T 1314a2e9bc9f4ffe84f1e1b933899c2e P 4a80a1ab47ee4291bc69d1719068abed: This TxnStatusTablet replica's current role is: LEADER
I20260807 15:11:28.183209 449 tablet_replica.cc:439] T 1314a2e9bc9f4ffe84f1e1b933899c2e P 4a80a1ab47ee4291bc69d1719068abed: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "4a80a1ab47ee4291bc69d1719068abed" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62c4c4f74c3744ddbc461ef9b6acfad4" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 35187 } } peers { permanent_uuid: "4a80a1ab47ee4291bc69d1719068abed" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 37143 } } peers { permanent_uuid: "4e1cf8eab0004fa8a9101887482ef407" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 34479 } } }
I20260807 15:11:28.183259 449 tablet_replica.cc:442] T 1314a2e9bc9f4ffe84f1e1b933899c2e P 4a80a1ab47ee4291bc69d1719068abed: This TxnStatusTablet replica's current role is: LEADER
I20260807 15:11:28.183632 396 tablet_replica.cc:439] T 1314a2e9bc9f4ffe84f1e1b933899c2e P 4e1cf8eab0004fa8a9101887482ef407: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "4a80a1ab47ee4291bc69d1719068abed" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62c4c4f74c3744ddbc461ef9b6acfad4" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 35187 } } peers { permanent_uuid: "4a80a1ab47ee4291bc69d1719068abed" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 37143 } } peers { permanent_uuid: "4e1cf8eab0004fa8a9101887482ef407" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 34479 } } }
I20260807 15:11:28.183718 396 tablet_replica.cc:442] T 1314a2e9bc9f4ffe84f1e1b933899c2e P 4e1cf8eab0004fa8a9101887482ef407: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 15:11:28.183842 392 tablet_replica.cc:439] T 1314a2e9bc9f4ffe84f1e1b933899c2e P 62c4c4f74c3744ddbc461ef9b6acfad4: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "4a80a1ab47ee4291bc69d1719068abed" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62c4c4f74c3744ddbc461ef9b6acfad4" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 35187 } } peers { permanent_uuid: "4a80a1ab47ee4291bc69d1719068abed" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 37143 } } peers { permanent_uuid: "4e1cf8eab0004fa8a9101887482ef407" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 34479 } } }
I20260807 15:11:28.183923 392 tablet_replica.cc:442] T 1314a2e9bc9f4ffe84f1e1b933899c2e P 62c4c4f74c3744ddbc461ef9b6acfad4: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 15:11:28.191859 479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:28.188412 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.2:37143 (local address 127.29.194.1:48941)
0807 15:11:28.189593 (+ 1181us) negotiation.cc:107] Waiting for socket to connect
0807 15:11:28.189605 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 15:11:28.189707 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:11:28.189835 (+ 128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:11:28.189835 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:11:28.189837 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 15:11:28.189893 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:28.189894 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:28.190533 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:28.190533 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:28.191092 (+ 559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:28.191094 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:28.191472 (+ 378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:28.191473 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:28.191520 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:28.191537 (+ 17us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 15:11:28.191692 (+ 155us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 15:11:28.191694 (+ 2us) client_negotiation.cc:769] Sending connection context
0807 15:11:28.191730 (+ 36us) client_negotiation.cc:240] Negotiation successful
0807 15:11:28.191779 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1132,"thread_start_us":96,"threads_started":1}
I20260807 15:11:28.191854 462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:28.188516 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.194.1:48941 (local address 127.29.194.2:37143)
0807 15:11:28.188582 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:28.188584 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:28.189673 (+ 1089us) server_negotiation.cc:407] Connection header received
0807 15:11:28.189778 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:28.189780 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:28.189791 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:28.189833 (+ 42us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 15:11:28.189968 (+ 135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:28.190458 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:28.191195 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:28.191388 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:28.191443 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:28.191609 (+ 166us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 15:11:28.191646 (+ 37us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 15:11:28.191686 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0807 15:11:28.191741 (+ 55us) server_negotiation.cc:299] Negotiation successful
0807 15:11:28.191765 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260807 15:11:28.194644 32743 raft_consensus.cc:1261] T 386d5751599d43fe95c17a123a960efe P 4e1cf8eab0004fa8a9101887482ef407 [term 1 FOLLOWER]: Refusing update from remote peer 62c4c4f74c3744ddbc461ef9b6acfad4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:11:28.194645 32527 raft_consensus.cc:1261] T 386d5751599d43fe95c17a123a960efe P 4a80a1ab47ee4291bc69d1719068abed [term 1 FOLLOWER]: Refusing update from remote peer 62c4c4f74c3744ddbc461ef9b6acfad4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:11:28.194887 417 consensus_queue.cc:1035] T 386d5751599d43fe95c17a123a960efe P 62c4c4f74c3744ddbc461ef9b6acfad4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e1cf8eab0004fa8a9101887482ef407" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 34479 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:11:28.195014 392 consensus_queue.cc:1035] T 386d5751599d43fe95c17a123a960efe P 62c4c4f74c3744ddbc461ef9b6acfad4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4a80a1ab47ee4291bc69d1719068abed" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 37143 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:11:28.208313 382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:28.201759 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.194.2:53999 (local address 127.29.194.1:35187)
0807 15:11:28.201802 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:28.201804 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:28.205696 (+ 3892us) server_negotiation.cc:407] Connection header received
0807 15:11:28.205763 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:28.205764 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:28.205777 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:28.205832 (+ 55us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 15:11:28.206042 (+ 210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:28.206529 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:28.207640 (+ 1111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:28.207833 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:28.207890 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:28.208088 (+ 198us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 15:11:28.208116 (+ 28us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 15:11:28.208148 (+ 32us) server_negotiation.cc:1035] Waiting for connection context
0807 15:11:28.208215 (+ 67us) server_negotiation.cc:299] Negotiation successful
0807 15:11:28.208236 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260807 15:11:28.208369 486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:28.201704 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.1:35187 (local address 127.29.194.2:53999)
0807 15:11:28.205606 (+ 3902us) negotiation.cc:107] Waiting for socket to connect
0807 15:11:28.205621 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 15:11:28.205700 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:11:28.205845 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:11:28.205846 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:11:28.205892 (+ 46us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 15:11:28.205976 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:28.205978 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:28.206603 (+ 625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:28.206604 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:28.207544 (+ 940us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:28.207548 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:28.207909 (+ 361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:28.207910 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:28.207984 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:28.208007 (+ 23us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 15:11:28.208163 (+ 156us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 15:11:28.208165 (+ 2us) client_negotiation.cc:769] Sending connection context
0807 15:11:28.208213 (+ 48us) client_negotiation.cc:240] Negotiation successful
0807 15:11:28.208261 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3838,"thread_start_us":88,"threads_started":1}
I20260807 15:11:28.218845 32105 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43848:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 15:11:28.218973 32105 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43848:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 15:11:28.219933 32105 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b676891f78094012accc7df16ace058c: Sending DeleteTablet for 3 replicas of tablet 386d5751599d43fe95c17a123a960efe
I20260807 15:11:28.220304 32499 tablet_service.cc:1514] Processing DeleteTablet for tablet 386d5751599d43fe95c17a123a960efe with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:11:28 UTC) from {username='kudu', principal='kudu/127.29.194.62@KRBTEST.COM'} at 127.0.0.1:42626
I20260807 15:11:28.220419 32710 tablet_service.cc:1514] Processing DeleteTablet for tablet 386d5751599d43fe95c17a123a960efe with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:11:28 UTC) from {username='kudu', principal='kudu/127.29.194.62@KRBTEST.COM'} at 127.0.0.1:34038
I20260807 15:11:28.220672 488 tablet_replica.cc:330] T 386d5751599d43fe95c17a123a960efe P 4a80a1ab47ee4291bc69d1719068abed: stopping tablet replica
I20260807 15:11:28.220729 488 raft_consensus.cc:2229] T 386d5751599d43fe95c17a123a960efe P 4a80a1ab47ee4291bc69d1719068abed [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 15:11:28.220690 32297 tablet_service.cc:1514] Processing DeleteTablet for tablet 386d5751599d43fe95c17a123a960efe with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:11:28 UTC) from {username='kudu', principal='kudu/127.29.194.62@KRBTEST.COM'} at 127.0.0.1:34140
I20260807 15:11:28.220769 488 raft_consensus.cc:2258] T 386d5751599d43fe95c17a123a960efe P 4a80a1ab47ee4291bc69d1719068abed [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:11:28.220902 494 tablet_replica.cc:330] T 386d5751599d43fe95c17a123a960efe P 62c4c4f74c3744ddbc461ef9b6acfad4: stopping tablet replica
I20260807 15:11:28.220956 494 raft_consensus.cc:2229] T 386d5751599d43fe95c17a123a960efe P 62c4c4f74c3744ddbc461ef9b6acfad4 [term 1 LEADER]: Raft consensus shutting down.
I20260807 15:11:28.220973 489 tablet_replica.cc:330] T 386d5751599d43fe95c17a123a960efe P 4e1cf8eab0004fa8a9101887482ef407: stopping tablet replica
I20260807 15:11:28.221024 494 raft_consensus.cc:2258] T 386d5751599d43fe95c17a123a960efe P 62c4c4f74c3744ddbc461ef9b6acfad4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:11:28.221026 489 raft_consensus.cc:2229] T 386d5751599d43fe95c17a123a960efe P 4e1cf8eab0004fa8a9101887482ef407 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 15:11:28.221064 489 raft_consensus.cc:2258] T 386d5751599d43fe95c17a123a960efe P 4e1cf8eab0004fa8a9101887482ef407 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:11:28.221323 494 ts_tablet_manager.cc:1905] T 386d5751599d43fe95c17a123a960efe P 62c4c4f74c3744ddbc461ef9b6acfad4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:11:28.221330 489 ts_tablet_manager.cc:1905] T 386d5751599d43fe95c17a123a960efe P 4e1cf8eab0004fa8a9101887482ef407: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:11:28.223182 489 ts_tablet_manager.cc:1918] T 386d5751599d43fe95c17a123a960efe P 4e1cf8eab0004fa8a9101887482ef407: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 15:11:28.223248 489 log.cc:1196] T 386d5751599d43fe95c17a123a960efe P 4e1cf8eab0004fa8a9101887482ef407: Deleting WAL directory at /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115480851507-30472-0/minicluster-data/ts-2/wal/wals/386d5751599d43fe95c17a123a960efe
I20260807 15:11:28.223232 494 ts_tablet_manager.cc:1918] T 386d5751599d43fe95c17a123a960efe P 62c4c4f74c3744ddbc461ef9b6acfad4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 15:11:28.223284 494 log.cc:1196] T 386d5751599d43fe95c17a123a960efe P 62c4c4f74c3744ddbc461ef9b6acfad4: Deleting WAL directory at /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115480851507-30472-0/minicluster-data/ts-0/wal/wals/386d5751599d43fe95c17a123a960efe
I20260807 15:11:28.223472 489 ts_tablet_manager.cc:1939] T 386d5751599d43fe95c17a123a960efe P 4e1cf8eab0004fa8a9101887482ef407: Deleting consensus metadata
I20260807 15:11:28.223526 494 ts_tablet_manager.cc:1939] T 386d5751599d43fe95c17a123a960efe P 62c4c4f74c3744ddbc461ef9b6acfad4: Deleting consensus metadata
I20260807 15:11:28.223838 32082 catalog_manager.cc:4841] TS 4e1cf8eab0004fa8a9101887482ef407 (127.29.194.3:34479): tablet 386d5751599d43fe95c17a123a960efe (table test-table [id=1bd1c73d1bed4b36a332fa0172946d1b]) successfully deleted
I20260807 15:11:28.223860 32081 catalog_manager.cc:4841] TS 62c4c4f74c3744ddbc461ef9b6acfad4 (127.29.194.1:35187): tablet 386d5751599d43fe95c17a123a960efe (table test-table [id=1bd1c73d1bed4b36a332fa0172946d1b]) successfully deleted
I20260807 15:11:28.224172 488 ts_tablet_manager.cc:1905] T 386d5751599d43fe95c17a123a960efe P 4a80a1ab47ee4291bc69d1719068abed: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:11:28.226076 488 ts_tablet_manager.cc:1918] T 386d5751599d43fe95c17a123a960efe P 4a80a1ab47ee4291bc69d1719068abed: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 15:11:28.226145 488 log.cc:1196] T 386d5751599d43fe95c17a123a960efe P 4a80a1ab47ee4291bc69d1719068abed: Deleting WAL directory at /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115480851507-30472-0/minicluster-data/ts-1/wal/wals/386d5751599d43fe95c17a123a960efe
I20260807 15:11:28.226364 488 ts_tablet_manager.cc:1939] T 386d5751599d43fe95c17a123a960efe P 4a80a1ab47ee4291bc69d1719068abed: Deleting consensus metadata
I20260807 15:11:28.226842 32081 catalog_manager.cc:4841] TS 4a80a1ab47ee4291bc69d1719068abed (127.29.194.2:37143): tablet 386d5751599d43fe95c17a123a960efe (table test-table [id=1bd1c73d1bed4b36a332fa0172946d1b]) successfully deleted
Aug 07 15:11:28 dist-test-slave-42z9 krb5kdc[32035](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115487, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.29.194.1@KRBTEST.COM
I20260807 15:11:28.231069 382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:28.221676 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34156 (local address 127.29.194.1:35187)
0807 15:11:28.221719 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:28.221721 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:28.221952 (+ 231us) server_negotiation.cc:407] Connection header received
0807 15:11:28.222038 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:28.222039 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:28.222051 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:28.222079 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:11:28.222799 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:28.223487 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:28.224529 (+ 1042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:28.224731 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:28.224786 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:28.227375 (+ 2589us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:11:28.227388 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:11:28.227388 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:11:28.227397 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:11:28.229244 (+ 1847us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:28.229949 (+ 705us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:28.229953 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:28.229953 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:28.230037 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:28.230337 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:28.230338 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:28.230338 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:28.230536 (+ 198us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:11:28.230696 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0807 15:11:28.230861 (+ 165us) server_negotiation.cc:299] Negotiation successful
0807 15:11:28.230968 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
W20260807 15:11:28.231402 32387 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34156
I20260807 15:11:28.239488 32170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:28.232455 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43870 (local address 127.29.194.62:45747)
0807 15:11:28.232516 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:28.232518 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:28.232530 (+ 12us) server_negotiation.cc:407] Connection header received
0807 15:11:28.232576 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:28.232577 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:28.232588 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:28.232650 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:11:28.233111 (+ 461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:28.233888 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:28.234600 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:28.234885 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:28.234955 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:28.236307 (+ 1352us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:11:28.236309 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:11:28.236309 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:11:28.236318 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:11:28.238277 (+ 1959us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:28.238720 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:28.238721 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:28.238722 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:28.238782 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:28.238956 (+ 174us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:28.238957 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:28.238957 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:28.239085 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:11:28.239183 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0807 15:11:28.239292 (+ 109us) server_negotiation.cc:299] Negotiation successful
0807 15:11:28.239393 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
W20260807 15:11:28.239729 32105 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43870
I20260807 15:11:28.240243 30472 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7fZSIL/build/release/bin/kudu with pid 32201
I20260807 15:11:28.259452 30472 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7fZSIL/build/release/bin/kudu with pid 32406
I20260807 15:11:28.274838 30472 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7fZSIL/build/release/bin/kudu with pid 32611
I20260807 15:11:28.285553 30472 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7fZSIL/build/release/bin/kudu with pid 32058
2026-08-07T15:11:28Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (2858 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115480851507-30472-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 15:11:28 dist-test-slave-42z9 krb5kdc[502](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 15:11:28 dist-test-slave-42z9 krb5kdc[502](info): set up 2 sockets
Aug 07 15:11:28 dist-test-slave-42z9 krb5kdc[502](info): commencing operation
krb5kdc: starting...
W20260807 15:11:29.452111 30472 mini_kdc.cc:121] Time spent starting KDC: real 1.137s user 0.003s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115480851507-30472-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115480851507-30472-0/krb5kdc/test-user.keytab.
Aug 07 15:11:29 dist-test-slave-42z9 krb5kdc[502](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115489, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T15:11:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T15:11:29Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.194.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.194.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.194.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.62.keytab.
Entry for principal kudu/127.29.194.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.62.keytab.
I20260807 15:11:29.597517 30472 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7fZSIL/build/release/bin/kudu
/tmp/dist-test-task7fZSIL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115480851507-30472-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115480851507-30472-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115480851507-30472-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115480851507-30472-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.194.62:43411
--webserver_interface=127.29.194.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.62.keytab
--principal=kudu/127.29.194.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.194.20:34009
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.194.62:43411
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115480851507-30472-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115480851507-30472-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115480851507-30472-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:11:29.685173 525 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:11:29.685474 525 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:11:29.685514 525 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:11:29.686782 525 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:11:29.686836 525 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260807 15:11:29.686852 525 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:11:29.686869 525 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:11:29.686887 525 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:11:29.686901 525 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:11:29.688694 525 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.194.20:34009
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115480851507-30472-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115480851507-30472-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.194.62:43411
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.62.keytab
--principal=kudu/127.29.194.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.194.62:43411
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115480851507-30472-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.194.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.525
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115480851507-30472-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:11:29.688984 525 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:11:29.689200 525 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:11:29.694494 542 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:11:29.694481 536 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:11:29.694658 525 server_base.cc:1034] running on GCE node
W20260807 15:11:29.694481 538 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:11:29.695482 525 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:11:29.695818 525 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:11:29.697021 525 hybrid_clock.cc:648] HybridClock initialized: now 1786115489696978 us; error 70 us; skew 500 ppm
Aug 07 15:11:29 dist-test-slave-42z9 krb5kdc[502](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115489, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.194.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:11:29.700577 525 init.cc:377] Logged in from keytab as kudu/127.29.194.62@KRBTEST.COM (short username kudu)
I20260807 15:11:29.701192 525 webserver.cc:460] Webserver started at http://127.29.194.62:33195/ using document root <none> and password file <none>
I20260807 15:11:29.701519 525 fs_manager.cc:359] Metadata directory not provided
I20260807 15:11:29.701581 525 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:11:29.701738 525 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:11:29.703001 525 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115480851507-30472-0/minicluster-data/master-0/data/instance:
uuid: "86993e10419047948fdd2437d6f0c4fc"
format_stamp: "Formatted at 2026-08-07 15:11:29 on dist-test-slave-42z9"
server_key: "d27332f9ae78de363922f95c7fb8be3a"
server_key_iv: "af5cc2dd149cfe2862cc887cab0fbc72"
server_key_version: "encryptionkey@0"
I20260807 15:11:29.703409 525 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115480851507-30472-0/minicluster-data/master-0/wal/instance:
uuid: "86993e10419047948fdd2437d6f0c4fc"
format_stamp: "Formatted at 2026-08-07 15:11:29 on dist-test-slave-42z9"
server_key: "d27332f9ae78de363922f95c7fb8be3a"
server_key_iv: "af5cc2dd149cfe2862cc887cab0fbc72"
server_key_version: "encryptionkey@0"
I20260807 15:11:29.705111 525 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.001s
I20260807 15:11:29.706341 552 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:11:29.706672 525 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 15:11:29.706805 525 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115480851507-30472-0/minicluster-data/master-0/data,/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115480851507-30472-0/minicluster-data/master-0/wal
uuid: "86993e10419047948fdd2437d6f0c4fc"
format_stamp: "Formatted at 2026-08-07 15:11:29 on dist-test-slave-42z9"
server_key: "d27332f9ae78de363922f95c7fb8be3a"
server_key_iv: "af5cc2dd149cfe2862cc887cab0fbc72"
server_key_version: "encryptionkey@0"
I20260807 15:11:29.706920 525 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115480851507-30472-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115480851507-30472-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115480851507-30472-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:11:29.734637 525 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:11:29.737157 525 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:11:29.737316 525 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:11:29.742479 525 rpc_server.cc:307] RPC server started. Bound to: 127.29.194.62:43411
I20260807 15:11:29.742537 651 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.194.62:43411 every 8 connection(s)
I20260807 15:11:29.742902 525 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115480851507-30472-0/minicluster-data/master-0/data/info.pb
I20260807 15:11:29.743916 652 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:11:29.744458 30472 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7fZSIL/build/release/bin/kudu as pid 525
I20260807 15:11:29.744566 30472 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115480851507-30472-0/minicluster-data/master-0/wal/instance
I20260807 15:11:29.744922 30472 external_mini_cluster.cc:1428] Setting key f85918d38452f41c1308d37655929410
I20260807 15:11:29.747473 652 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 86993e10419047948fdd2437d6f0c4fc: Bootstrap starting.
I20260807 15:11:29.748129 652 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 86993e10419047948fdd2437d6f0c4fc: Neither blocks nor log segments found. Creating new log.
I20260807 15:11:29.748418 652 log.cc:824] T 00000000000000000000000000000000 P 86993e10419047948fdd2437d6f0c4fc: Log is configured to *not* fsync() on all Append() calls
I20260807 15:11:29.749107 652 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 86993e10419047948fdd2437d6f0c4fc: No bootstrap required, opened a new log
I20260807 15:11:29.751223 652 raft_consensus.cc:348] T 00000000000000000000000000000000 P 86993e10419047948fdd2437d6f0c4fc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86993e10419047948fdd2437d6f0c4fc" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 43411 } }
I20260807 15:11:29.751394 652 raft_consensus.cc:374] T 00000000000000000000000000000000 P 86993e10419047948fdd2437d6f0c4fc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:11:29.751435 652 raft_consensus.cc:729] T 00000000000000000000000000000000 P 86993e10419047948fdd2437d6f0c4fc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 86993e10419047948fdd2437d6f0c4fc, State: Initialized, Role: FOLLOWER
I20260807 15:11:29.751575 652 consensus_queue.cc:260] T 00000000000000000000000000000000 P 86993e10419047948fdd2437d6f0c4fc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86993e10419047948fdd2437d6f0c4fc" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 43411 } }
I20260807 15:11:29.751662 652 raft_consensus.cc:388] T 00000000000000000000000000000000 P 86993e10419047948fdd2437d6f0c4fc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:11:29.751698 652 raft_consensus.cc:482] T 00000000000000000000000000000000 P 86993e10419047948fdd2437d6f0c4fc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:11:29.751744 652 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 86993e10419047948fdd2437d6f0c4fc [term 0 FOLLOWER]: Advancing to term 1
Aug 07 15:11:29 dist-test-slave-42z9 krb5kdc[502](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115489, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.194.62@KRBTEST.COM
I20260807 15:11:29.752450 652 raft_consensus.cc:504] T 00000000000000000000000000000000 P 86993e10419047948fdd2437d6f0c4fc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86993e10419047948fdd2437d6f0c4fc" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 43411 } }
I20260807 15:11:29.752550 652 leader_election.cc:302] T 00000000000000000000000000000000 P 86993e10419047948fdd2437d6f0c4fc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 86993e10419047948fdd2437d6f0c4fc; no voters:
I20260807 15:11:29.752748 652 leader_election.cc:288] T 00000000000000000000000000000000 P 86993e10419047948fdd2437d6f0c4fc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 15:11:29.752800 660 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 86993e10419047948fdd2437d6f0c4fc [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:11:29.752974 660 raft_consensus.cc:686] T 00000000000000000000000000000000 P 86993e10419047948fdd2437d6f0c4fc [term 1 LEADER]: Becoming Leader. State: Replica: 86993e10419047948fdd2437d6f0c4fc, State: Running, Role: LEADER
I20260807 15:11:29.753059 652 sys_catalog.cc:564] T 00000000000000000000000000000000 P 86993e10419047948fdd2437d6f0c4fc [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:11:29.753079 660 consensus_queue.cc:237] T 00000000000000000000000000000000 P 86993e10419047948fdd2437d6f0c4fc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86993e10419047948fdd2437d6f0c4fc" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 43411 } }
I20260807 15:11:29.753424 662 sys_catalog.cc:455] T 00000000000000000000000000000000 P 86993e10419047948fdd2437d6f0c4fc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "86993e10419047948fdd2437d6f0c4fc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86993e10419047948fdd2437d6f0c4fc" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 43411 } } }
I20260807 15:11:29.753553 662 sys_catalog.cc:458] T 00000000000000000000000000000000 P 86993e10419047948fdd2437d6f0c4fc [sys.catalog]: This master's current role is: LEADER
I20260807 15:11:29.753414 660 sys_catalog.cc:455] T 00000000000000000000000000000000 P 86993e10419047948fdd2437d6f0c4fc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 86993e10419047948fdd2437d6f0c4fc. Latest consensus state: current_term: 1 leader_uuid: "86993e10419047948fdd2437d6f0c4fc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86993e10419047948fdd2437d6f0c4fc" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 43411 } } }
I20260807 15:11:29.753655 660 sys_catalog.cc:458] T 00000000000000000000000000000000 P 86993e10419047948fdd2437d6f0c4fc [sys.catalog]: This master's current role is: LEADER
I20260807 15:11:29.755060 666 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:11:29.756243 666 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:11:29.757357 658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:29.745809 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40180 (local address 127.29.194.62:43411)
0807 15:11:29.746100 (+ 291us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:29.746110 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:29.746161 (+ 51us) server_negotiation.cc:407] Connection header received
0807 15:11:29.746637 (+ 476us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:29.746644 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:29.746697 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:29.746862 (+ 165us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:11:29.747572 (+ 710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:29.748605 (+ 1033us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:29.749386 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:29.749861 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:29.749976 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:29.752612 (+ 2636us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:11:29.752618 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:11:29.752618 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:11:29.752632 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:11:29.754863 (+ 2231us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:29.755499 (+ 636us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:29.755502 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:29.755503 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:29.755643 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:29.755961 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:29.755962 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:29.755962 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:29.756296 (+ 334us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:11:29.756469 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0807 15:11:29.756860 (+ 391us) server_negotiation.cc:299] Negotiation successful
0807 15:11:29.757101 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":226,"thread_start_us":139,"threads_started":1}
I20260807 15:11:29.758311 666 catalog_manager.cc:1347] Generated new cluster ID: 4b74442760cd4fc0a314f8fcbbf76762
I20260807 15:11:29.758381 666 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:11:29.772670 666 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:11:29.773370 666 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:11:29.785650 666 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 86993e10419047948fdd2437d6f0c4fc: Generated new TSK 0
I20260807 15:11:29.785858 666 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.29.194.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.194.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.194.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.1.keytab.
Entry for principal kudu/127.29.194.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.1.keytab.
I20260807 15:11:29.817845 30472 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7fZSIL/build/release/bin/kudu
/tmp/dist-test-task7fZSIL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115480851507-30472-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115480851507-30472-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115480851507-30472-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115480851507-30472-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.194.1:0
--local_ip_for_outbound_sockets=127.29.194.1
--webserver_interface=127.29.194.1
--webserver_port=0
--tserver_master_addrs=127.29.194.62:43411
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.1.keytab
--principal=kudu/127.29.194.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.194.20:34009
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115480851507-30472-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115480851507-30472-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115480851507-30472-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:11:29.910102 687 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:11:29.910333 687 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:11:29.910374 687 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:11:29.911667 687 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:11:29.911720 687 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:11:29.911749 687 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260807 15:11:29.911764 687 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 15:11:29.911790 687 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.194.1
I20260807 15:11:29.913477 687 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.194.20:34009
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115480851507-30472-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115480851507-30472-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.1.keytab
--principal=kudu/127.29.194.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.194.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115480851507-30472-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.194.1
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.29.194.62:43411
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.687
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.194.1
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115480851507-30472-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:11:29.913746 687 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:11:29.913983 687 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:11:29.919520 694 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:11:29.919520 692 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:11:29.919593 696 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:11:29.920014 687 server_base.cc:1034] running on GCE node
I20260807 15:11:29.920252 687 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:11:29.920511 687 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:11:29.921657 687 hybrid_clock.cc:648] HybridClock initialized: now 1786115489921646 us; error 37 us; skew 500 ppm
Aug 07 15:11:29 dist-test-slave-42z9 krb5kdc[502](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115489, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.194.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:11:29.925261 687 init.cc:377] Logged in from keytab as kudu/127.29.194.1@KRBTEST.COM (short username kudu)
I20260807 15:11:29.925915 687 webserver.cc:460] Webserver started at http://127.29.194.1:43221/ using document root <none> and password file <none>
I20260807 15:11:29.926182 687 fs_manager.cc:359] Metadata directory not provided
I20260807 15:11:29.926230 687 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:11:29.926366 687 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:11:29.927656 687 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115480851507-30472-0/minicluster-data/ts-0/data/instance:
uuid: "0cd27c0909644c38991b15cd2c5bc574"
format_stamp: "Formatted at 2026-08-07 15:11:29 on dist-test-slave-42z9"
server_key: "07e32c0f4d725544ceda86cb1f87c4fd"
server_key_iv: "76cfa07a526e51c74e49185a4d8ecc0f"
server_key_version: "encryptionkey@0"
I20260807 15:11:29.928035 687 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115480851507-30472-0/minicluster-data/ts-0/wal/instance:
uuid: "0cd27c0909644c38991b15cd2c5bc574"
format_stamp: "Formatted at 2026-08-07 15:11:29 on dist-test-slave-42z9"
server_key: "07e32c0f4d725544ceda86cb1f87c4fd"
server_key_iv: "76cfa07a526e51c74e49185a4d8ecc0f"
server_key_version: "encryptionkey@0"
I20260807 15:11:29.929890 687 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.003s
I20260807 15:11:29.931111 710 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:11:29.931353 687 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:11:29.931455 687 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115480851507-30472-0/minicluster-data/ts-0/data,/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115480851507-30472-0/minicluster-data/ts-0/wal
uuid: "0cd27c0909644c38991b15cd2c5bc574"
format_stamp: "Formatted at 2026-08-07 15:11:29 on dist-test-slave-42z9"
server_key: "07e32c0f4d725544ceda86cb1f87c4fd"
server_key_iv: "76cfa07a526e51c74e49185a4d8ecc0f"
server_key_version: "encryptionkey@0"
I20260807 15:11:29.931547 687 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115480851507-30472-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115480851507-30472-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115480851507-30472-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:11:29.945580 687 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:11:29.947793 687 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:11:29.947918 687 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:11:29.948439 687 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:11:29.948477 687 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:11:29.948500 687 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:11:29.948513 687 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:11:29.955149 687 rpc_server.cc:307] RPC server started. Bound to: 127.29.194.1:34761
I20260807 15:11:29.955324 877 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.194.1:34761 every 8 connection(s)
I20260807 15:11:29.955610 687 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115480851507-30472-0/minicluster-data/ts-0/data/info.pb
Aug 07 15:11:29 dist-test-slave-42z9 krb5kdc[502](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115489, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.194.1@KRBTEST.COM for kudu/127.29.194.62@KRBTEST.COM
I20260807 15:11:29.959542 658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:29.949043 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.194.1:42211 (local address 127.29.194.62:43411)
0807 15:11:29.949095 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:29.949097 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:29.949341 (+ 244us) server_negotiation.cc:407] Connection header received
0807 15:11:29.949957 (+ 616us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:29.949959 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:29.949973 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:29.950043 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:11:29.950908 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:29.951692 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:29.953310 (+ 1618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:29.953694 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:29.953771 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:29.956709 (+ 2938us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:11:29.956711 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:11:29.956711 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:11:29.956721 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:11:29.958098 (+ 1377us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:29.958549 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:29.958550 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:29.958551 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:29.958609 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:29.958848 (+ 239us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:29.958849 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:29.958849 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:29.958981 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:11:29.959038 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0807 15:11:29.959324 (+ 286us) server_negotiation.cc:299] Negotiation successful
0807 15:11:29.959443 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260807 15:11:29.959734 726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:29.949039 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:43411 (local address 127.29.194.1:42211)
0807 15:11:29.949207 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0807 15:11:29.949242 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0807 15:11:29.949837 (+ 595us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:11:29.950120 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:11:29.950125 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:11:29.950488 (+ 363us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:11:29.950789 (+ 301us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:29.950795 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:29.951796 (+ 1001us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:29.951797 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:29.953142 (+ 1345us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:29.953147 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:29.953805 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:29.953806 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:29.954016 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:29.954692 (+ 676us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:11:29.954700 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:11:29.956595 (+ 1895us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:11:29.958195 (+ 1600us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:11:29.958207 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:11:29.958210 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:11:29.958473 (+ 263us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:11:29.958667 (+ 194us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:11:29.958668 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:11:29.958668 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:11:29.958790 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:11:29.959042 (+ 252us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:11:29.959049 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:11:29.959200 (+ 151us) client_negotiation.cc:769] Sending connection context
0807 15:11:29.959294 (+ 94us) client_negotiation.cc:240] Negotiation successful
0807 15:11:29.959497 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":90,"threads_started":1}
I20260807 15:11:29.960270 878 heartbeater.cc:347] Connected to a master server at 127.29.194.62:43411
I20260807 15:11:29.960357 878 heartbeater.cc:464] Registering TS with master...
I20260807 15:11:29.960558 878 heartbeater.cc:511] Master 127.29.194.62:43411 requested a full tablet report, sending...
I20260807 15:11:29.961170 572 ts_manager.cc:194] Registered new tserver with Master: 0cd27c0909644c38991b15cd2c5bc574 (127.29.194.1:34761)
I20260807 15:11:29.962121 572 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.194.1@KRBTEST.COM'} at 127.29.194.1:42211
I20260807 15:11:29.965652 30472 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7fZSIL/build/release/bin/kudu as pid 687
I20260807 15:11:29.965762 30472 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115480851507-30472-0/minicluster-data/ts-0/wal/instance
I20260807 15:11:29.965957 30472 external_mini_cluster.cc:1428] Setting key 2dc9062567587f6ee4f0ace135adeed7
I20260807 15:11:29.967895 891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:29.961416 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:43411 (local address 127.29.194.1:41137)
0807 15:11:29.961575 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0807 15:11:29.961587 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 15:11:29.961653 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:11:29.961775 (+ 122us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:11:29.961776 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:11:29.961994 (+ 218us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:11:29.962076 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:29.962079 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:29.962911 (+ 832us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:29.962912 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:29.963431 (+ 519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:29.963433 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:29.963870 (+ 437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:29.963873 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:29.964002 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:29.964710 (+ 708us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:11:29.964715 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:11:29.965054 (+ 339us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:11:29.966935 (+ 1881us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:11:29.966936 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:11:29.966937 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:11:29.967127 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:11:29.967328 (+ 201us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:11:29.967328 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:11:29.967329 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:11:29.967378 (+ 49us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:11:29.967620 (+ 242us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:11:29.967620 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:11:29.967666 (+ 46us) client_negotiation.cc:769] Sending connection context
0807 15:11:29.967717 (+ 51us) client_negotiation.cc:240] Negotiation successful
0807 15:11:29.967820 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":96,"threads_started":1}
I20260807 15:11:29.967927 658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:29.961524 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.194.1:41137 (local address 127.29.194.62:43411)
0807 15:11:29.961571 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:29.961573 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:29.961637 (+ 64us) server_negotiation.cc:407] Connection header received
0807 15:11:29.961704 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:29.961705 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:29.961714 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:29.961757 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:11:29.962144 (+ 387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:29.962809 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:29.963516 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:29.963706 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:29.963767 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:29.965186 (+ 1419us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:11:29.965187 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:11:29.965187 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:11:29.965195 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:11:29.966818 (+ 1623us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:29.967201 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:29.967202 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:29.967203 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:29.967267 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:29.967424 (+ 157us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:29.967425 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:29.967425 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:29.967559 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:11:29.967620 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0807 15:11:29.967752 (+ 132us) server_negotiation.cc:299] Negotiation successful
0807 15:11:29.967838 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
WARNING: no policy specified for kudu/127.29.194.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.194.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.194.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.2.keytab.
Entry for principal kudu/127.29.194.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.2.keytab.
I20260807 15:11:29.997304 30472 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7fZSIL/build/release/bin/kudu
/tmp/dist-test-task7fZSIL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115480851507-30472-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115480851507-30472-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115480851507-30472-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115480851507-30472-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.194.2:0
--local_ip_for_outbound_sockets=127.29.194.2
--webserver_interface=127.29.194.2
--webserver_port=0
--tserver_master_addrs=127.29.194.62:43411
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.2.keytab
--principal=kudu/127.29.194.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.194.20:34009
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115480851507-30472-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115480851507-30472-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115480851507-30472-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:11:30.084792 895 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:11:30.085023 895 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:11:30.085049 895 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:11:30.086346 895 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:11:30.086436 895 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:11:30.086458 895 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260807 15:11:30.086468 895 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 15:11:30.086485 895 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.194.2
I20260807 15:11:30.088078 895 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.194.20:34009
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115480851507-30472-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115480851507-30472-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.2.keytab
--principal=kudu/127.29.194.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.194.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115480851507-30472-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.194.2
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.29.194.62:43411
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.895
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.194.2
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115480851507-30472-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:11:30.088364 895 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:11:30.088608 895 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:11:30.093799 908 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:11:30.093918 907 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:11:30.093802 910 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:11:30.093863 895 server_base.cc:1034] running on GCE node
I20260807 15:11:30.094151 895 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:11:30.094419 895 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:11:30.095587 895 hybrid_clock.cc:648] HybridClock initialized: now 1786115490095560 us; error 44 us; skew 500 ppm
Aug 07 15:11:30 dist-test-slave-42z9 krb5kdc[502](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115490, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.194.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:11:30.098387 895 init.cc:377] Logged in from keytab as kudu/127.29.194.2@KRBTEST.COM (short username kudu)
I20260807 15:11:30.098868 895 webserver.cc:460] Webserver started at http://127.29.194.2:39489/ using document root <none> and password file <none>
I20260807 15:11:30.099076 895 fs_manager.cc:359] Metadata directory not provided
I20260807 15:11:30.099128 895 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:11:30.099258 895 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:11:30.100653 895 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115480851507-30472-0/minicluster-data/ts-1/data/instance:
uuid: "5a1ab2d4360447afa9f51e9c649e3eb1"
format_stamp: "Formatted at 2026-08-07 15:11:30 on dist-test-slave-42z9"
server_key: "04610935156ae8ab4fe91b7dd0aba5b5"
server_key_iv: "97942619bb1c7b811a72964ed4497969"
server_key_version: "encryptionkey@0"
I20260807 15:11:30.101280 895 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115480851507-30472-0/minicluster-data/ts-1/wal/instance:
uuid: "5a1ab2d4360447afa9f51e9c649e3eb1"
format_stamp: "Formatted at 2026-08-07 15:11:30 on dist-test-slave-42z9"
server_key: "04610935156ae8ab4fe91b7dd0aba5b5"
server_key_iv: "97942619bb1c7b811a72964ed4497969"
server_key_version: "encryptionkey@0"
I20260807 15:11:30.102953 895 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260807 15:11:30.103776 920 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:11:30.103982 895 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 15:11:30.104084 895 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115480851507-30472-0/minicluster-data/ts-1/data,/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115480851507-30472-0/minicluster-data/ts-1/wal
uuid: "5a1ab2d4360447afa9f51e9c649e3eb1"
format_stamp: "Formatted at 2026-08-07 15:11:30 on dist-test-slave-42z9"
server_key: "04610935156ae8ab4fe91b7dd0aba5b5"
server_key_iv: "97942619bb1c7b811a72964ed4497969"
server_key_version: "encryptionkey@0"
I20260807 15:11:30.104166 895 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115480851507-30472-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115480851507-30472-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115480851507-30472-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:11:30.114961 895 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:11:30.117165 895 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:11:30.117286 895 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:11:30.117861 895 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:11:30.117906 895 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:11:30.117935 895 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:11:30.117952 895 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:11:30.125267 895 rpc_server.cc:307] RPC server started. Bound to: 127.29.194.2:37537
I20260807 15:11:30.125458 1091 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.194.2:37537 every 8 connection(s)
Aug 07 15:11:30 dist-test-slave-42z9 krb5kdc[502](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115490, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.194.2@KRBTEST.COM for kudu/127.29.194.62@KRBTEST.COM
I20260807 15:11:30.125734 895 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115480851507-30472-0/minicluster-data/ts-1/data/info.pb
I20260807 15:11:30.129173 658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:30.118317 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.194.2:58587 (local address 127.29.194.62:43411)
0807 15:11:30.118362 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:30.118364 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:30.118617 (+ 253us) server_negotiation.cc:407] Connection header received
0807 15:11:30.119142 (+ 525us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:30.119142 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:30.119156 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:30.119217 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:11:30.120175 (+ 958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:30.120977 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:30.122634 (+ 1657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:30.123052 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:30.123189 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:30.126610 (+ 3421us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:11:30.126613 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:11:30.126613 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:11:30.126621 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:11:30.127818 (+ 1197us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:30.128222 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:30.128222 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:30.128223 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:30.128271 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:30.128570 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:30.128571 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:30.128572 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:30.128725 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:11:30.128779 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0807 15:11:30.129023 (+ 244us) server_negotiation.cc:299] Negotiation successful
0807 15:11:30.129106 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260807 15:11:30.129377 935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:30.118314 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:43411 (local address 127.29.194.2:58587)
0807 15:11:30.118495 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0807 15:11:30.118532 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0807 15:11:30.119041 (+ 509us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:11:30.119354 (+ 313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:11:30.119359 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:11:30.119740 (+ 381us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:11:30.120048 (+ 308us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:30.120054 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:30.121086 (+ 1032us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:30.121087 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:30.122478 (+ 1391us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:30.122482 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:30.123202 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:30.123203 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:30.123366 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:30.124100 (+ 734us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:11:30.124112 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:11:30.126498 (+ 2386us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:11:30.127902 (+ 1404us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:11:30.127909 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:11:30.127911 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:11:30.128161 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:11:30.128372 (+ 211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:11:30.128373 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:11:30.128373 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:11:30.128500 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:11:30.128777 (+ 277us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:11:30.128780 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:11:30.128906 (+ 126us) client_negotiation.cc:769] Sending connection context
0807 15:11:30.129004 (+ 98us) client_negotiation.cc:240] Negotiation successful
0807 15:11:30.129150 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":105,"threads_started":1}
I20260807 15:11:30.129925 1092 heartbeater.cc:347] Connected to a master server at 127.29.194.62:43411
I20260807 15:11:30.130012 1092 heartbeater.cc:464] Registering TS with master...
I20260807 15:11:30.130203 1092 heartbeater.cc:511] Master 127.29.194.62:43411 requested a full tablet report, sending...
I20260807 15:11:30.130580 571 ts_manager.cc:194] Registered new tserver with Master: 5a1ab2d4360447afa9f51e9c649e3eb1 (127.29.194.2:37537)
I20260807 15:11:30.131103 571 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.194.2@KRBTEST.COM'} at 127.29.194.2:58587
I20260807 15:11:30.134712 30472 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7fZSIL/build/release/bin/kudu as pid 895
I20260807 15:11:30.134855 30472 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115480851507-30472-0/minicluster-data/ts-1/wal/instance
I20260807 15:11:30.135097 30472 external_mini_cluster.cc:1428] Setting key 2e4b231f3f40c28165c33157fa818f9f
I20260807 15:11:30.138096 1102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:30.130926 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:43411 (local address 127.29.194.2:37657)
0807 15:11:30.131066 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0807 15:11:30.131077 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 15:11:30.131144 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:11:30.131283 (+ 139us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:11:30.131284 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:11:30.131467 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:11:30.131538 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:30.131540 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:30.132429 (+ 889us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:30.132431 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:30.133113 (+ 682us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:30.133119 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:30.133565 (+ 446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:30.133566 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:30.133645 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:30.134080 (+ 435us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:11:30.134085 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:11:30.134419 (+ 334us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:11:30.136734 (+ 2315us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:11:30.136735 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:11:30.136735 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:11:30.136950 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:11:30.137311 (+ 361us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:11:30.137312 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:11:30.137312 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:11:30.137387 (+ 75us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:11:30.137804 (+ 417us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:11:30.137805 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:11:30.137856 (+ 51us) client_negotiation.cc:769] Sending connection context
0807 15:11:30.137906 (+ 50us) client_negotiation.cc:240] Negotiation successful
0807 15:11:30.138014 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"mutex_wait_us":2,"thread_start_us":84,"threads_started":1}
I20260807 15:11:30.138145 658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:30.131014 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.194.2:37657 (local address 127.29.194.62:43411)
0807 15:11:30.131057 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:30.131059 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:30.131129 (+ 70us) server_negotiation.cc:407] Connection header received
0807 15:11:30.131194 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:30.131195 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:30.131207 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:30.131264 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:11:30.131617 (+ 353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:30.132221 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:30.133251 (+ 1030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:30.133488 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:30.133560 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:30.134535 (+ 975us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:11:30.134536 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:11:30.134537 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:11:30.134544 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:11:30.136622 (+ 2078us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:30.137043 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:30.137045 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:30.137045 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:30.137209 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:30.137498 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:30.137499 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:30.137500 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:30.137729 (+ 229us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:11:30.137802 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0807 15:11:30.137951 (+ 149us) server_negotiation.cc:299] Negotiation successful
0807 15:11:30.138057 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
WARNING: no policy specified for kudu/127.29.194.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.194.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.194.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.3.keytab.
Entry for principal kudu/127.29.194.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.3.keytab.
I20260807 15:11:30.168883 30472 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7fZSIL/build/release/bin/kudu
/tmp/dist-test-task7fZSIL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115480851507-30472-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115480851507-30472-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115480851507-30472-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115480851507-30472-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.194.3:0
--local_ip_for_outbound_sockets=127.29.194.3
--webserver_interface=127.29.194.3
--webserver_port=0
--tserver_master_addrs=127.29.194.62:43411
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.3.keytab
--principal=kudu/127.29.194.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.194.20:34009
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115480851507-30472-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115480851507-30472-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115480851507-30472-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:11:30.258483 1106 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:11:30.258720 1106 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:11:30.258749 1106 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:11:30.260040 1106 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:11:30.260106 1106 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:11:30.260129 1106 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260807 15:11:30.260140 1106 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 15:11:30.260159 1106 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.194.3
I20260807 15:11:30.261897 1106 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.194.20:34009
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115480851507-30472-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115480851507-30472-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.3.keytab
--principal=kudu/127.29.194.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.194.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115480851507-30472-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.194.3
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.29.194.62:43411
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1106
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.194.3
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115480851507-30472-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:11:30.262159 1106 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:11:30.262399 1106 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 15:11:30.267940 1106 server_base.cc:1034] running on GCE node
W20260807 15:11:30.267906 1112 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:11:30.267916 1114 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:11:30.267916 1117 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:11:30.268354 1106 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:11:30.268637 1106 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:11:30.269807 1106 hybrid_clock.cc:648] HybridClock initialized: now 1786115490269744 us; error 79 us; skew 500 ppm
Aug 07 15:11:30 dist-test-slave-42z9 krb5kdc[502](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115490, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.194.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:11:30.272859 1106 init.cc:377] Logged in from keytab as kudu/127.29.194.3@KRBTEST.COM (short username kudu)
I20260807 15:11:30.273350 1106 webserver.cc:460] Webserver started at http://127.29.194.3:44085/ using document root <none> and password file <none>
I20260807 15:11:30.273646 1106 fs_manager.cc:359] Metadata directory not provided
I20260807 15:11:30.273705 1106 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:11:30.273833 1106 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:11:30.274922 1106 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115480851507-30472-0/minicluster-data/ts-2/data/instance:
uuid: "cf43c478316e4b46a30860f95d7997b2"
format_stamp: "Formatted at 2026-08-07 15:11:30 on dist-test-slave-42z9"
server_key: "827653ded620205880394cc44d3e8b4a"
server_key_iv: "7791acd036d6e0c2f445a5e6b3c6bede"
server_key_version: "encryptionkey@0"
I20260807 15:11:30.275274 1106 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115480851507-30472-0/minicluster-data/ts-2/wal/instance:
uuid: "cf43c478316e4b46a30860f95d7997b2"
format_stamp: "Formatted at 2026-08-07 15:11:30 on dist-test-slave-42z9"
server_key: "827653ded620205880394cc44d3e8b4a"
server_key_iv: "7791acd036d6e0c2f445a5e6b3c6bede"
server_key_version: "encryptionkey@0"
I20260807 15:11:30.276868 1106 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 15:11:30.277909 1125 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:11:30.278298 1106 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:11:30.278455 1106 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115480851507-30472-0/minicluster-data/ts-2/data,/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115480851507-30472-0/minicluster-data/ts-2/wal
uuid: "cf43c478316e4b46a30860f95d7997b2"
format_stamp: "Formatted at 2026-08-07 15:11:30 on dist-test-slave-42z9"
server_key: "827653ded620205880394cc44d3e8b4a"
server_key_iv: "7791acd036d6e0c2f445a5e6b3c6bede"
server_key_version: "encryptionkey@0"
I20260807 15:11:30.278719 1106 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115480851507-30472-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115480851507-30472-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115480851507-30472-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:11:30.296860 1106 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:11:30.299464 1106 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:11:30.299638 1106 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:11:30.300248 1106 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:11:30.300292 1106 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:11:30.300325 1106 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:11:30.300349 1106 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:11:30.307957 1106 rpc_server.cc:307] RPC server started. Bound to: 127.29.194.3:33731
I20260807 15:11:30.308125 1295 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.194.3:33731 every 8 connection(s)
Aug 07 15:11:30 dist-test-slave-42z9 krb5kdc[502](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115490, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.194.3@KRBTEST.COM for kudu/127.29.194.62@KRBTEST.COM
I20260807 15:11:30.308388 1106 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115480851507-30472-0/minicluster-data/ts-2/data/info.pb
I20260807 15:11:30.311658 658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:30.300990 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.194.3:46735 (local address 127.29.194.62:43411)
0807 15:11:30.301074 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:30.301077 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:30.301322 (+ 245us) server_negotiation.cc:407] Connection header received
0807 15:11:30.302053 (+ 731us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:30.302056 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:30.302073 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:30.302144 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:11:30.303200 (+ 1056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:30.303949 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:30.305587 (+ 1638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:30.305886 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:30.305968 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:30.308901 (+ 2933us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:11:30.308903 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:11:30.308903 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:11:30.308911 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:11:30.310180 (+ 1269us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:30.310640 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:30.310640 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:30.310641 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:30.310689 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:30.310923 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:30.310924 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:30.310924 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:30.311104 (+ 180us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:11:30.311165 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0807 15:11:30.311512 (+ 347us) server_negotiation.cc:299] Negotiation successful
0807 15:11:30.311591 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260807 15:11:30.311805 1140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:30.300923 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:43411 (local address 127.29.194.3:46735)
0807 15:11:30.301152 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0807 15:11:30.301203 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0807 15:11:30.301883 (+ 680us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:11:30.302259 (+ 376us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:11:30.302265 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:11:30.302717 (+ 452us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:11:30.303076 (+ 359us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:30.303082 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:30.304079 (+ 997us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:30.304081 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:30.305404 (+ 1323us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:30.305411 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:30.305982 (+ 571us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:30.305983 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:30.306137 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:30.306777 (+ 640us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:11:30.306788 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:11:30.308796 (+ 2008us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:11:30.310311 (+ 1515us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:11:30.310319 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:11:30.310321 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:11:30.310575 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:11:30.310742 (+ 167us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:11:30.310742 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:11:30.310743 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:11:30.310841 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:11:30.311168 (+ 327us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:11:30.311175 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:11:30.311326 (+ 151us) client_negotiation.cc:769] Sending connection context
0807 15:11:30.311454 (+ 128us) client_negotiation.cc:240] Negotiation successful
0807 15:11:30.311605 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":130,"threads_started":1}
I20260807 15:11:30.312314 1297 heartbeater.cc:347] Connected to a master server at 127.29.194.62:43411
I20260807 15:11:30.312408 1297 heartbeater.cc:464] Registering TS with master...
I20260807 15:11:30.312600 1297 heartbeater.cc:511] Master 127.29.194.62:43411 requested a full tablet report, sending...
I20260807 15:11:30.312994 572 ts_manager.cc:194] Registered new tserver with Master: cf43c478316e4b46a30860f95d7997b2 (127.29.194.3:33731)
I20260807 15:11:30.313484 572 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.194.3@KRBTEST.COM'} at 127.29.194.3:46735
I20260807 15:11:30.316412 30472 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7fZSIL/build/release/bin/kudu as pid 1106
I20260807 15:11:30.316507 30472 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115480851507-30472-0/minicluster-data/ts-2/wal/instance
I20260807 15:11:30.316679 30472 external_mini_cluster.cc:1428] Setting key a85c79f4fc0a0a72aa1366ee6714a160
I20260807 15:11:30.318042 30472 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 15:11:30.320706 1308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:30.313631 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:43411 (local address 127.29.194.3:58509)
0807 15:11:30.313837 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0807 15:11:30.313848 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 15:11:30.313911 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:11:30.314089 (+ 178us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:11:30.314090 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:11:30.314297 (+ 207us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:11:30.314360 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:30.314363 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:30.315136 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:30.315137 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:30.315780 (+ 643us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:30.315783 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:30.316243 (+ 460us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:30.316244 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:30.316338 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:30.317119 (+ 781us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:11:30.317126 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:11:30.317584 (+ 458us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:11:30.319533 (+ 1949us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:11:30.319534 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:11:30.319535 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:11:30.319763 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:11:30.320046 (+ 283us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:11:30.320047 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:11:30.320047 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:11:30.320112 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:11:30.320432 (+ 320us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:11:30.320433 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:11:30.320479 (+ 46us) client_negotiation.cc:769] Sending connection context
0807 15:11:30.320528 (+ 49us) client_negotiation.cc:240] Negotiation successful
0807 15:11:30.320631 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":122,"threads_started":1}
I20260807 15:11:30.320761 658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:30.313656 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.194.3:58509 (local address 127.29.194.62:43411)
0807 15:11:30.313699 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:30.313701 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:30.313928 (+ 227us) server_negotiation.cc:407] Connection header received
0807 15:11:30.313969 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:30.313970 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:30.313988 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:30.314058 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:11:30.314502 (+ 444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:30.314994 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:30.315893 (+ 899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:30.316111 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:30.316183 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:30.317670 (+ 1487us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:11:30.317671 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:11:30.317672 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:11:30.317680 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:11:30.319429 (+ 1749us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:30.319894 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:30.319895 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:30.319895 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:30.319971 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:30.320173 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:30.320173 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:30.320174 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:30.320370 (+ 196us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:11:30.320447 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0807 15:11:30.320567 (+ 120us) server_negotiation.cc:299] Negotiation successful
0807 15:11:30.320666 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
Aug 07 15:11:30 dist-test-slave-42z9 krb5kdc[502](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115490, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 07 15:11:30 dist-test-slave-42z9 krb5kdc[502](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115490, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.194.62@KRBTEST.COM
I20260807 15:11:30.346858 658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:30.337722 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40188 (local address 127.29.194.62:43411)
0807 15:11:30.337780 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:30.337782 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:30.337945 (+ 163us) server_negotiation.cc:407] Connection header received
0807 15:11:30.337979 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:30.337980 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:30.337992 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:30.338061 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:11:30.338868 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:30.339440 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:30.340332 (+ 892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:30.340578 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:30.340748 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:30.343628 (+ 2880us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:11:30.343630 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:11:30.343630 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:11:30.343642 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:11:30.345404 (+ 1762us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:30.346001 (+ 597us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:30.346003 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:30.346003 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:30.346078 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:30.346387 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:30.346388 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:30.346388 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:30.346527 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:11:30.346620 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0807 15:11:30.346708 (+ 88us) server_negotiation.cc:299] Negotiation successful
0807 15:11:30.346785 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260807 15:11:30.349228 572 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40188:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 15:11:30.351037 572 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 15:11:30.358932 1326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:30.353903 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.3:33731 (local address 127.0.0.1:55238)
0807 15:11:30.354205 (+ 302us) negotiation.cc:107] Waiting for socket to connect
0807 15:11:30.354266 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0807 15:11:30.354397 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:11:30.355042 (+ 645us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:11:30.355046 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:11:30.355048 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:11:30.355146 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:30.355148 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:30.356189 (+ 1041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:30.356191 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:30.357348 (+ 1157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:30.357351 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:30.358452 (+ 1101us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:30.358453 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:30.358569 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:30.358587 (+ 18us) client_negotiation.cc:769] Sending connection context
0807 15:11:30.358738 (+ 151us) client_negotiation.cc:240] Negotiation successful
0807 15:11:30.358822 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":114,"threads_started":1}
I20260807 15:11:30.359002 1329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:30.354056 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55238 (local address 127.29.194.3:33731)
0807 15:11:30.354826 (+ 770us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:30.354830 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:30.354840 (+ 10us) server_negotiation.cc:407] Connection header received
0807 15:11:30.354882 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:30.354884 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:30.354900 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:30.355026 (+ 126us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:11:30.355261 (+ 235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:30.356094 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:30.357495 (+ 1401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:30.358358 (+ 863us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:30.358439 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:30.358517 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0807 15:11:30.358765 (+ 248us) server_negotiation.cc:299] Negotiation successful
0807 15:11:30.358819 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":732,"thread_start_us":129,"threads_started":1}
I20260807 15:11:30.359408 1333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:30.354579 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53104 (local address 127.29.194.1:34761)
0807 15:11:30.354756 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:30.354761 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:30.355022 (+ 261us) server_negotiation.cc:407] Connection header received
0807 15:11:30.355064 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:30.355066 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:30.355082 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:30.355165 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:11:30.355458 (+ 293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:30.356659 (+ 1201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:30.358048 (+ 1389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:30.358725 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:30.358795 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:30.358854 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0807 15:11:30.359230 (+ 376us) server_negotiation.cc:299] Negotiation successful
0807 15:11:30.359282 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":140,"thread_start_us":97,"threads_started":1}
I20260807 15:11:30.359303 1330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:30.354198 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.2:37537 (local address 127.0.0.1:52208)
0807 15:11:30.354349 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0807 15:11:30.354364 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 15:11:30.354443 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:11:30.355212 (+ 769us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:11:30.355213 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:11:30.355215 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:11:30.355312 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:30.355315 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:30.356770 (+ 1455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:30.356772 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:30.357959 (+ 1187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:30.357963 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:30.358962 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:30.358964 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:30.359061 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:30.359067 (+ 6us) client_negotiation.cc:769] Sending connection context
0807 15:11:30.359107 (+ 40us) client_negotiation.cc:240] Negotiation successful
0807 15:11:30.359162 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":104,"threads_started":1}
I20260807 15:11:30.359848 1331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:30.354215 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.1:34761 (local address 127.0.0.1:53104)
0807 15:11:30.354886 (+ 671us) negotiation.cc:107] Waiting for socket to connect
0807 15:11:30.354897 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 15:11:30.354953 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:11:30.355257 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:11:30.355258 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:11:30.355261 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:11:30.355348 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:30.355351 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:30.356764 (+ 1413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:30.356765 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:30.357917 (+ 1152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:30.357921 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:30.358890 (+ 969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:30.358891 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:30.358970 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:30.358976 (+ 6us) client_negotiation.cc:769] Sending connection context
0807 15:11:30.359023 (+ 47us) client_negotiation.cc:240] Negotiation successful
0807 15:11:30.359080 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":647,"thread_start_us":162,"threads_started":1}
I20260807 15:11:30.359812 1332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:30.354536 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52208 (local address 127.29.194.2:37537)
0807 15:11:30.354747 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:30.354753 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:30.354766 (+ 13us) server_negotiation.cc:407] Connection header received
0807 15:11:30.354853 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:30.354855 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:30.354876 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:30.355376 (+ 500us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:11:30.355441 (+ 65us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:30.356652 (+ 1211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:30.358066 (+ 1414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:30.358747 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:30.359387 (+ 640us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:30.359473 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0807 15:11:30.359502 (+ 29us) server_negotiation.cc:299] Negotiation successful
0807 15:11:30.359564 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":170,"thread_start_us":130,"threads_started":1}
I20260807 15:11:30.360486 1205 tablet_service.cc:1467] Processing CreateTablet for tablet 0267dcc371874c90a7649fd82dde29a7 (DEFAULT_TABLE table=test-table [id=a2ab468749944c7292cfeff59d4f5d24]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:11:30.360842 998 tablet_service.cc:1467] Processing CreateTablet for tablet 0267dcc371874c90a7649fd82dde29a7 (DEFAULT_TABLE table=test-table [id=a2ab468749944c7292cfeff59d4f5d24]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:11:30.361090 1205 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0267dcc371874c90a7649fd82dde29a7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:11:30.360908 781 tablet_service.cc:1467] Processing CreateTablet for tablet 0267dcc371874c90a7649fd82dde29a7 (DEFAULT_TABLE table=test-table [id=a2ab468749944c7292cfeff59d4f5d24]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:11:30.361491 998 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0267dcc371874c90a7649fd82dde29a7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:11:30.361636 781 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0267dcc371874c90a7649fd82dde29a7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:11:30.366370 1339 tablet_bootstrap.cc:492] T 0267dcc371874c90a7649fd82dde29a7 P cf43c478316e4b46a30860f95d7997b2: Bootstrap starting.
I20260807 15:11:30.366593 1337 tablet_bootstrap.cc:492] T 0267dcc371874c90a7649fd82dde29a7 P 0cd27c0909644c38991b15cd2c5bc574: Bootstrap starting.
I20260807 15:11:30.367209 1337 tablet_bootstrap.cc:654] T 0267dcc371874c90a7649fd82dde29a7 P 0cd27c0909644c38991b15cd2c5bc574: Neither blocks nor log segments found. Creating new log.
I20260807 15:11:30.367300 1339 tablet_bootstrap.cc:654] T 0267dcc371874c90a7649fd82dde29a7 P cf43c478316e4b46a30860f95d7997b2: Neither blocks nor log segments found. Creating new log.
I20260807 15:11:30.367530 1337 log.cc:824] T 0267dcc371874c90a7649fd82dde29a7 P 0cd27c0909644c38991b15cd2c5bc574: Log is configured to *not* fsync() on all Append() calls
I20260807 15:11:30.367612 1339 log.cc:824] T 0267dcc371874c90a7649fd82dde29a7 P cf43c478316e4b46a30860f95d7997b2: Log is configured to *not* fsync() on all Append() calls
I20260807 15:11:30.367955 1340 tablet_bootstrap.cc:492] T 0267dcc371874c90a7649fd82dde29a7 P 5a1ab2d4360447afa9f51e9c649e3eb1: Bootstrap starting.
I20260807 15:11:30.368306 1337 tablet_bootstrap.cc:492] T 0267dcc371874c90a7649fd82dde29a7 P 0cd27c0909644c38991b15cd2c5bc574: No bootstrap required, opened a new log
I20260807 15:11:30.368367 1337 ts_tablet_manager.cc:1397] T 0267dcc371874c90a7649fd82dde29a7 P 0cd27c0909644c38991b15cd2c5bc574: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 15:11:30.368525 1340 tablet_bootstrap.cc:654] T 0267dcc371874c90a7649fd82dde29a7 P 5a1ab2d4360447afa9f51e9c649e3eb1: Neither blocks nor log segments found. Creating new log.
I20260807 15:11:30.368784 1340 log.cc:824] T 0267dcc371874c90a7649fd82dde29a7 P 5a1ab2d4360447afa9f51e9c649e3eb1: Log is configured to *not* fsync() on all Append() calls
I20260807 15:11:30.369436 1340 tablet_bootstrap.cc:492] T 0267dcc371874c90a7649fd82dde29a7 P 5a1ab2d4360447afa9f51e9c649e3eb1: No bootstrap required, opened a new log
I20260807 15:11:30.369539 1340 ts_tablet_manager.cc:1397] T 0267dcc371874c90a7649fd82dde29a7 P 5a1ab2d4360447afa9f51e9c649e3eb1: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 15:11:30.369941 1339 tablet_bootstrap.cc:492] T 0267dcc371874c90a7649fd82dde29a7 P cf43c478316e4b46a30860f95d7997b2: No bootstrap required, opened a new log
I20260807 15:11:30.370009 1339 ts_tablet_manager.cc:1397] T 0267dcc371874c90a7649fd82dde29a7 P cf43c478316e4b46a30860f95d7997b2: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260807 15:11:30.370301 1337 raft_consensus.cc:348] T 0267dcc371874c90a7649fd82dde29a7 P 0cd27c0909644c38991b15cd2c5bc574 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf43c478316e4b46a30860f95d7997b2" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 33731 } } peers { permanent_uuid: "5a1ab2d4360447afa9f51e9c649e3eb1" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 37537 } } peers { permanent_uuid: "0cd27c0909644c38991b15cd2c5bc574" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 34761 } }
I20260807 15:11:30.370491 1337 raft_consensus.cc:374] T 0267dcc371874c90a7649fd82dde29a7 P 0cd27c0909644c38991b15cd2c5bc574 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:11:30.370535 1337 raft_consensus.cc:729] T 0267dcc371874c90a7649fd82dde29a7 P 0cd27c0909644c38991b15cd2c5bc574 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0cd27c0909644c38991b15cd2c5bc574, State: Initialized, Role: FOLLOWER
I20260807 15:11:30.370790 1337 consensus_queue.cc:260] T 0267dcc371874c90a7649fd82dde29a7 P 0cd27c0909644c38991b15cd2c5bc574 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf43c478316e4b46a30860f95d7997b2" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 33731 } } peers { permanent_uuid: "5a1ab2d4360447afa9f51e9c649e3eb1" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 37537 } } peers { permanent_uuid: "0cd27c0909644c38991b15cd2c5bc574" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 34761 } }
I20260807 15:11:30.371083 1337 ts_tablet_manager.cc:1428] T 0267dcc371874c90a7649fd82dde29a7 P 0cd27c0909644c38991b15cd2c5bc574: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260807 15:11:30.371143 878 heartbeater.cc:503] Master 127.29.194.62:43411 was elected leader, sending a full tablet report...
I20260807 15:11:30.371886 1339 raft_consensus.cc:348] T 0267dcc371874c90a7649fd82dde29a7 P cf43c478316e4b46a30860f95d7997b2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf43c478316e4b46a30860f95d7997b2" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 33731 } } peers { permanent_uuid: "5a1ab2d4360447afa9f51e9c649e3eb1" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 37537 } } peers { permanent_uuid: "0cd27c0909644c38991b15cd2c5bc574" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 34761 } }
I20260807 15:11:30.372051 1339 raft_consensus.cc:374] T 0267dcc371874c90a7649fd82dde29a7 P cf43c478316e4b46a30860f95d7997b2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:11:30.372093 1339 raft_consensus.cc:729] T 0267dcc371874c90a7649fd82dde29a7 P cf43c478316e4b46a30860f95d7997b2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf43c478316e4b46a30860f95d7997b2, State: Initialized, Role: FOLLOWER
I20260807 15:11:30.372227 1339 consensus_queue.cc:260] T 0267dcc371874c90a7649fd82dde29a7 P cf43c478316e4b46a30860f95d7997b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf43c478316e4b46a30860f95d7997b2" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 33731 } } peers { permanent_uuid: "5a1ab2d4360447afa9f51e9c649e3eb1" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 37537 } } peers { permanent_uuid: "0cd27c0909644c38991b15cd2c5bc574" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 34761 } }
I20260807 15:11:30.372524 1339 ts_tablet_manager.cc:1428] T 0267dcc371874c90a7649fd82dde29a7 P cf43c478316e4b46a30860f95d7997b2: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260807 15:11:30.372646 1297 heartbeater.cc:503] Master 127.29.194.62:43411 was elected leader, sending a full tablet report...
I20260807 15:11:30.373215 1340 raft_consensus.cc:348] T 0267dcc371874c90a7649fd82dde29a7 P 5a1ab2d4360447afa9f51e9c649e3eb1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf43c478316e4b46a30860f95d7997b2" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 33731 } } peers { permanent_uuid: "5a1ab2d4360447afa9f51e9c649e3eb1" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 37537 } } peers { permanent_uuid: "0cd27c0909644c38991b15cd2c5bc574" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 34761 } }
I20260807 15:11:30.373389 1340 raft_consensus.cc:374] T 0267dcc371874c90a7649fd82dde29a7 P 5a1ab2d4360447afa9f51e9c649e3eb1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:11:30.373420 1340 raft_consensus.cc:729] T 0267dcc371874c90a7649fd82dde29a7 P 5a1ab2d4360447afa9f51e9c649e3eb1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a1ab2d4360447afa9f51e9c649e3eb1, State: Initialized, Role: FOLLOWER
I20260807 15:11:30.373560 1340 consensus_queue.cc:260] T 0267dcc371874c90a7649fd82dde29a7 P 5a1ab2d4360447afa9f51e9c649e3eb1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf43c478316e4b46a30860f95d7997b2" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 33731 } } peers { permanent_uuid: "5a1ab2d4360447afa9f51e9c649e3eb1" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 37537 } } peers { permanent_uuid: "0cd27c0909644c38991b15cd2c5bc574" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 34761 } }
I20260807 15:11:30.373859 1340 ts_tablet_manager.cc:1428] T 0267dcc371874c90a7649fd82dde29a7 P 5a1ab2d4360447afa9f51e9c649e3eb1: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260807 15:11:30.373914 1092 heartbeater.cc:503] Master 127.29.194.62:43411 was elected leader, sending a full tablet report...
W20260807 15:11:30.376595 1094 tablet.cc:2369] T 0267dcc371874c90a7649fd82dde29a7 P 5a1ab2d4360447afa9f51e9c649e3eb1: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 15:11:30.383451 1347 raft_consensus.cc:482] T 0267dcc371874c90a7649fd82dde29a7 P 5a1ab2d4360447afa9f51e9c649e3eb1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 15:11:30.383672 1347 raft_consensus.cc:504] T 0267dcc371874c90a7649fd82dde29a7 P 5a1ab2d4360447afa9f51e9c649e3eb1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf43c478316e4b46a30860f95d7997b2" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 33731 } } peers { permanent_uuid: "5a1ab2d4360447afa9f51e9c649e3eb1" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 37537 } } peers { permanent_uuid: "0cd27c0909644c38991b15cd2c5bc574" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 34761 } }
I20260807 15:11:30.384150 1347 leader_election.cc:288] T 0267dcc371874c90a7649fd82dde29a7 P 5a1ab2d4360447afa9f51e9c649e3eb1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cf43c478316e4b46a30860f95d7997b2 (127.29.194.3:33731), 0cd27c0909644c38991b15cd2c5bc574 (127.29.194.1:34761)
I20260807 15:11:30.387528 1333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:30.384539 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.194.2:38057 (local address 127.29.194.1:34761)
0807 15:11:30.384604 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:30.384606 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:30.384618 (+ 12us) server_negotiation.cc:407] Connection header received
0807 15:11:30.384657 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:30.384658 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:30.384671 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:30.384741 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:11:30.384923 (+ 182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:30.385719 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:30.386756 (+ 1037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:30.387145 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:30.387213 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:30.387224 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0807 15:11:30.387405 (+ 181us) server_negotiation.cc:299] Negotiation successful
0807 15:11:30.387456 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260807 15:11:30.387528 935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:30.384427 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.1:34761 (local address 127.29.194.2:38057)
0807 15:11:30.384485 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0807 15:11:30.384499 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 15:11:30.384545 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:11:30.384751 (+ 206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:11:30.384753 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:11:30.384756 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:11:30.384853 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:30.384855 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:30.385842 (+ 987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:30.385844 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:30.386657 (+ 813us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:30.386659 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:30.387220 (+ 561us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:30.387221 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:30.387338 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:30.387342 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 15:11:30.387390 (+ 48us) client_negotiation.cc:240] Negotiation successful
0807 15:11:30.387434 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33,"mutex_wait_us":1}
I20260807 15:11:30.387590 1329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:30.384547 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.194.2:45163 (local address 127.29.194.3:33731)
0807 15:11:30.384619 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:30.384621 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:30.384939 (+ 318us) server_negotiation.cc:407] Connection header received
0807 15:11:30.384981 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:30.384982 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:30.384991 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:30.385040 (+ 49us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:11:30.385413 (+ 373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:30.385998 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:30.386901 (+ 903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:30.387276 (+ 375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:30.387341 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:30.387350 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0807 15:11:30.387479 (+ 129us) server_negotiation.cc:299] Negotiation successful
0807 15:11:30.387517 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260807 15:11:30.387650 1349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:30.384427 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.3:33731 (local address 127.29.194.2:45163)
0807 15:11:30.384861 (+ 434us) negotiation.cc:107] Waiting for socket to connect
0807 15:11:30.384870 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0807 15:11:30.384930 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:11:30.385041 (+ 111us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:11:30.385042 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:11:30.385043 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:11:30.385236 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:30.385246 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:30.386084 (+ 838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:30.386085 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:30.386813 (+ 728us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:30.386815 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:30.387349 (+ 534us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:30.387350 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:30.387420 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:30.387423 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 15:11:30.387467 (+ 44us) client_negotiation.cc:240] Negotiation successful
0807 15:11:30.387515 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":417,"mutex_wait_us":221,"thread_start_us":143,"threads_started":1}
I20260807 15:11:30.387956 1229 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0267dcc371874c90a7649fd82dde29a7" candidate_uuid: "5a1ab2d4360447afa9f51e9c649e3eb1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf43c478316e4b46a30860f95d7997b2" is_pre_election: true
I20260807 15:11:30.388098 1229 raft_consensus.cc:2454] T 0267dcc371874c90a7649fd82dde29a7 P cf43c478316e4b46a30860f95d7997b2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a1ab2d4360447afa9f51e9c649e3eb1 in term 0.
I20260807 15:11:30.388315 817 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0267dcc371874c90a7649fd82dde29a7" candidate_uuid: "5a1ab2d4360447afa9f51e9c649e3eb1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0cd27c0909644c38991b15cd2c5bc574" is_pre_election: true
I20260807 15:11:30.388350 924 leader_election.cc:302] T 0267dcc371874c90a7649fd82dde29a7 P 5a1ab2d4360447afa9f51e9c649e3eb1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a1ab2d4360447afa9f51e9c649e3eb1, cf43c478316e4b46a30860f95d7997b2; no voters:
I20260807 15:11:30.388439 817 raft_consensus.cc:2454] T 0267dcc371874c90a7649fd82dde29a7 P 0cd27c0909644c38991b15cd2c5bc574 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a1ab2d4360447afa9f51e9c649e3eb1 in term 0.
I20260807 15:11:30.388502 1347 raft_consensus.cc:2781] T 0267dcc371874c90a7649fd82dde29a7 P 5a1ab2d4360447afa9f51e9c649e3eb1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 15:11:30.388588 1347 raft_consensus.cc:482] T 0267dcc371874c90a7649fd82dde29a7 P 5a1ab2d4360447afa9f51e9c649e3eb1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 15:11:30.388628 1347 raft_consensus.cc:3037] T 0267dcc371874c90a7649fd82dde29a7 P 5a1ab2d4360447afa9f51e9c649e3eb1 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:11:30.389667 1347 raft_consensus.cc:504] T 0267dcc371874c90a7649fd82dde29a7 P 5a1ab2d4360447afa9f51e9c649e3eb1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf43c478316e4b46a30860f95d7997b2" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 33731 } } peers { permanent_uuid: "5a1ab2d4360447afa9f51e9c649e3eb1" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 37537 } } peers { permanent_uuid: "0cd27c0909644c38991b15cd2c5bc574" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 34761 } }
I20260807 15:11:30.389814 1347 leader_election.cc:288] T 0267dcc371874c90a7649fd82dde29a7 P 5a1ab2d4360447afa9f51e9c649e3eb1 [CANDIDATE]: Term 1 election: Requested vote from peers cf43c478316e4b46a30860f95d7997b2 (127.29.194.3:33731), 0cd27c0909644c38991b15cd2c5bc574 (127.29.194.1:34761)
I20260807 15:11:30.390007 817 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0267dcc371874c90a7649fd82dde29a7" candidate_uuid: "5a1ab2d4360447afa9f51e9c649e3eb1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0cd27c0909644c38991b15cd2c5bc574"
I20260807 15:11:30.390007 1229 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0267dcc371874c90a7649fd82dde29a7" candidate_uuid: "5a1ab2d4360447afa9f51e9c649e3eb1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf43c478316e4b46a30860f95d7997b2"
I20260807 15:11:30.390084 817 raft_consensus.cc:3037] T 0267dcc371874c90a7649fd82dde29a7 P 0cd27c0909644c38991b15cd2c5bc574 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:11:30.390085 1229 raft_consensus.cc:3037] T 0267dcc371874c90a7649fd82dde29a7 P cf43c478316e4b46a30860f95d7997b2 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:11:30.390825 817 raft_consensus.cc:2454] T 0267dcc371874c90a7649fd82dde29a7 P 0cd27c0909644c38991b15cd2c5bc574 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a1ab2d4360447afa9f51e9c649e3eb1 in term 1.
I20260807 15:11:30.390837 1229 raft_consensus.cc:2454] T 0267dcc371874c90a7649fd82dde29a7 P cf43c478316e4b46a30860f95d7997b2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a1ab2d4360447afa9f51e9c649e3eb1 in term 1.
I20260807 15:11:30.391002 924 leader_election.cc:302] T 0267dcc371874c90a7649fd82dde29a7 P 5a1ab2d4360447afa9f51e9c649e3eb1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a1ab2d4360447afa9f51e9c649e3eb1, cf43c478316e4b46a30860f95d7997b2; no voters:
I20260807 15:11:30.391155 1347 raft_consensus.cc:2781] T 0267dcc371874c90a7649fd82dde29a7 P 5a1ab2d4360447afa9f51e9c649e3eb1 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:11:30.391340 1347 raft_consensus.cc:686] T 0267dcc371874c90a7649fd82dde29a7 P 5a1ab2d4360447afa9f51e9c649e3eb1 [term 1 LEADER]: Becoming Leader. State: Replica: 5a1ab2d4360447afa9f51e9c649e3eb1, State: Running, Role: LEADER
I20260807 15:11:30.391431 1347 consensus_queue.cc:237] T 0267dcc371874c90a7649fd82dde29a7 P 5a1ab2d4360447afa9f51e9c649e3eb1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf43c478316e4b46a30860f95d7997b2" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 33731 } } peers { permanent_uuid: "5a1ab2d4360447afa9f51e9c649e3eb1" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 37537 } } peers { permanent_uuid: "0cd27c0909644c38991b15cd2c5bc574" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 34761 } }
I20260807 15:11:30.392272 572 catalog_manager.cc:5495] T 0267dcc371874c90a7649fd82dde29a7 P 5a1ab2d4360447afa9f51e9c649e3eb1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5a1ab2d4360447afa9f51e9c649e3eb1 (127.29.194.2). New cstate: current_term: 1 leader_uuid: "5a1ab2d4360447afa9f51e9c649e3eb1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf43c478316e4b46a30860f95d7997b2" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 33731 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5a1ab2d4360447afa9f51e9c649e3eb1" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 37537 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0cd27c0909644c38991b15cd2c5bc574" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 34761 } health_report { overall_health: UNKNOWN } } }
Aug 07 15:11:30 dist-test-slave-42z9 krb5kdc[502](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115489, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.194.62@KRBTEST.COM for kudu/127.29.194.62@KRBTEST.COM
I20260807 15:11:30.434871 1365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:30.426351 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:43411 (local address 127.0.0.1:40194)
0807 15:11:30.426541 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0807 15:11:30.426551 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 15:11:30.426610 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:11:30.426747 (+ 137us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:11:30.426747 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:11:30.427062 (+ 315us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:11:30.427120 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:30.427122 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:30.427864 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:30.427865 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:30.428413 (+ 548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:30.428414 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:30.428790 (+ 376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:30.428791 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:30.428844 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:30.429653 (+ 809us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:11:30.429658 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:11:30.431427 (+ 1769us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:11:30.433413 (+ 1986us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:11:30.433426 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:11:30.433428 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:11:30.433762 (+ 334us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:11:30.434034 (+ 272us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:11:30.434035 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:11:30.434035 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:11:30.434113 (+ 78us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:11:30.434518 (+ 405us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:11:30.434526 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:11:30.434577 (+ 51us) client_negotiation.cc:769] Sending connection context
0807 15:11:30.434642 (+ 65us) client_negotiation.cc:240] Negotiation successful
0807 15:11:30.434777 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":91,"threads_started":1}
I20260807 15:11:30.434976 658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:30.426500 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40194 (local address 127.29.194.62:43411)
0807 15:11:30.426555 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:30.426557 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:30.426598 (+ 41us) server_negotiation.cc:407] Connection header received
0807 15:11:30.426653 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:30.426654 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:30.426663 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:30.426734 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:11:30.427260 (+ 526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:30.427769 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:30.428514 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:30.428699 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:30.428755 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:30.431576 (+ 2821us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:11:30.431578 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:11:30.431578 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:11:30.431585 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:11:30.433276 (+ 1691us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:30.433850 (+ 574us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:30.433852 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:30.433852 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:30.433936 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:30.434238 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:30.434239 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:30.434239 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:30.434395 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:11:30.434465 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0807 15:11:30.434739 (+ 274us) server_negotiation.cc:299] Negotiation successful
0807 15:11:30.434843 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260807 15:11:30.437518 572 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.29.194.62@KRBTEST.COM'} at 127.0.0.1:40194:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260807 15:11:30.437891 572 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 15:11:30.439527 781 tablet_service.cc:1467] Processing CreateTablet for tablet d77090f464e944beb3386ce5dd2d256d (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=4020085e0dd74426bb94a8d8d8341311]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 15:11:30.439520 998 tablet_service.cc:1467] Processing CreateTablet for tablet d77090f464e944beb3386ce5dd2d256d (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=4020085e0dd74426bb94a8d8d8341311]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 15:11:30.439572 1205 tablet_service.cc:1467] Processing CreateTablet for tablet d77090f464e944beb3386ce5dd2d256d (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=4020085e0dd74426bb94a8d8d8341311]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 15:11:30.439707 1205 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d77090f464e944beb3386ce5dd2d256d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:11:30.439707 998 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d77090f464e944beb3386ce5dd2d256d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:11:30.439713 781 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d77090f464e944beb3386ce5dd2d256d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:11:30.441018 1337 tablet_bootstrap.cc:492] T d77090f464e944beb3386ce5dd2d256d P 0cd27c0909644c38991b15cd2c5bc574: Bootstrap starting.
I20260807 15:11:30.441213 1339 tablet_bootstrap.cc:492] T d77090f464e944beb3386ce5dd2d256d P cf43c478316e4b46a30860f95d7997b2: Bootstrap starting.
I20260807 15:11:30.441809 1337 tablet_bootstrap.cc:654] T d77090f464e944beb3386ce5dd2d256d P 0cd27c0909644c38991b15cd2c5bc574: Neither blocks nor log segments found. Creating new log.
I20260807 15:11:30.441854 1339 tablet_bootstrap.cc:654] T d77090f464e944beb3386ce5dd2d256d P cf43c478316e4b46a30860f95d7997b2: Neither blocks nor log segments found. Creating new log.
I20260807 15:11:30.442237 1340 tablet_bootstrap.cc:492] T d77090f464e944beb3386ce5dd2d256d P 5a1ab2d4360447afa9f51e9c649e3eb1: Bootstrap starting.
I20260807 15:11:30.442643 1340 tablet_bootstrap.cc:654] T d77090f464e944beb3386ce5dd2d256d P 5a1ab2d4360447afa9f51e9c649e3eb1: Neither blocks nor log segments found. Creating new log.
I20260807 15:11:30.442761 1339 tablet_bootstrap.cc:492] T d77090f464e944beb3386ce5dd2d256d P cf43c478316e4b46a30860f95d7997b2: No bootstrap required, opened a new log
I20260807 15:11:30.442822 1339 ts_tablet_manager.cc:1397] T d77090f464e944beb3386ce5dd2d256d P cf43c478316e4b46a30860f95d7997b2: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260807 15:11:30.443001 1339 raft_consensus.cc:348] T d77090f464e944beb3386ce5dd2d256d P cf43c478316e4b46a30860f95d7997b2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0cd27c0909644c38991b15cd2c5bc574" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 34761 } } peers { permanent_uuid: "5a1ab2d4360447afa9f51e9c649e3eb1" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 37537 } } peers { permanent_uuid: "cf43c478316e4b46a30860f95d7997b2" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 33731 } }
I20260807 15:11:30.443087 1339 raft_consensus.cc:374] T d77090f464e944beb3386ce5dd2d256d P cf43c478316e4b46a30860f95d7997b2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:11:30.443106 1339 raft_consensus.cc:729] T d77090f464e944beb3386ce5dd2d256d P cf43c478316e4b46a30860f95d7997b2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf43c478316e4b46a30860f95d7997b2, State: Initialized, Role: FOLLOWER
I20260807 15:11:30.443164 1339 consensus_queue.cc:260] T d77090f464e944beb3386ce5dd2d256d P cf43c478316e4b46a30860f95d7997b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0cd27c0909644c38991b15cd2c5bc574" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 34761 } } peers { permanent_uuid: "5a1ab2d4360447afa9f51e9c649e3eb1" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 37537 } } peers { permanent_uuid: "cf43c478316e4b46a30860f95d7997b2" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 33731 } }
I20260807 15:11:30.443254 1339 ts_tablet_manager.cc:1428] T d77090f464e944beb3386ce5dd2d256d P cf43c478316e4b46a30860f95d7997b2: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260807 15:11:30.443392 1346 tablet_replica.cc:439] T d77090f464e944beb3386ce5dd2d256d P cf43c478316e4b46a30860f95d7997b2: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0cd27c0909644c38991b15cd2c5bc574" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 34761 } } peers { permanent_uuid: "5a1ab2d4360447afa9f51e9c649e3eb1" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 37537 } } peers { permanent_uuid: "cf43c478316e4b46a30860f95d7997b2" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 33731 } } }
I20260807 15:11:30.443492 1346 tablet_replica.cc:442] T d77090f464e944beb3386ce5dd2d256d P cf43c478316e4b46a30860f95d7997b2: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 15:11:30.443591 1337 tablet_bootstrap.cc:492] T d77090f464e944beb3386ce5dd2d256d P 0cd27c0909644c38991b15cd2c5bc574: No bootstrap required, opened a new log
I20260807 15:11:30.443646 1337 ts_tablet_manager.cc:1397] T d77090f464e944beb3386ce5dd2d256d P 0cd27c0909644c38991b15cd2c5bc574: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260807 15:11:30.443830 1340 tablet_bootstrap.cc:492] T d77090f464e944beb3386ce5dd2d256d P 5a1ab2d4360447afa9f51e9c649e3eb1: No bootstrap required, opened a new log
I20260807 15:11:30.443794 1337 raft_consensus.cc:348] T d77090f464e944beb3386ce5dd2d256d P 0cd27c0909644c38991b15cd2c5bc574 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0cd27c0909644c38991b15cd2c5bc574" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 34761 } } peers { permanent_uuid: "5a1ab2d4360447afa9f51e9c649e3eb1" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 37537 } } peers { permanent_uuid: "cf43c478316e4b46a30860f95d7997b2" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 33731 } }
I20260807 15:11:30.443871 1337 raft_consensus.cc:374] T d77090f464e944beb3386ce5dd2d256d P 0cd27c0909644c38991b15cd2c5bc574 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:11:30.443882 1340 ts_tablet_manager.cc:1397] T d77090f464e944beb3386ce5dd2d256d P 5a1ab2d4360447afa9f51e9c649e3eb1: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 15:11:30.443892 1337 raft_consensus.cc:729] T d77090f464e944beb3386ce5dd2d256d P 0cd27c0909644c38991b15cd2c5bc574 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0cd27c0909644c38991b15cd2c5bc574, State: Initialized, Role: FOLLOWER
I20260807 15:11:30.443953 1337 consensus_queue.cc:260] T d77090f464e944beb3386ce5dd2d256d P 0cd27c0909644c38991b15cd2c5bc574 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0cd27c0909644c38991b15cd2c5bc574" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 34761 } } peers { permanent_uuid: "5a1ab2d4360447afa9f51e9c649e3eb1" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 37537 } } peers { permanent_uuid: "cf43c478316e4b46a30860f95d7997b2" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 33731 } }
I20260807 15:11:30.444041 1337 ts_tablet_manager.cc:1428] T d77090f464e944beb3386ce5dd2d256d P 0cd27c0909644c38991b15cd2c5bc574: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260807 15:11:30.444020 1340 raft_consensus.cc:348] T d77090f464e944beb3386ce5dd2d256d P 5a1ab2d4360447afa9f51e9c649e3eb1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0cd27c0909644c38991b15cd2c5bc574" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 34761 } } peers { permanent_uuid: "5a1ab2d4360447afa9f51e9c649e3eb1" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 37537 } } peers { permanent_uuid: "cf43c478316e4b46a30860f95d7997b2" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 33731 } }
I20260807 15:11:30.444103 1340 raft_consensus.cc:374] T d77090f464e944beb3386ce5dd2d256d P 5a1ab2d4360447afa9f51e9c649e3eb1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:11:30.444123 1340 raft_consensus.cc:729] T d77090f464e944beb3386ce5dd2d256d P 5a1ab2d4360447afa9f51e9c649e3eb1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a1ab2d4360447afa9f51e9c649e3eb1, State: Initialized, Role: FOLLOWER
I20260807 15:11:30.444169 1340 consensus_queue.cc:260] T d77090f464e944beb3386ce5dd2d256d P 5a1ab2d4360447afa9f51e9c649e3eb1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0cd27c0909644c38991b15cd2c5bc574" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 34761 } } peers { permanent_uuid: "5a1ab2d4360447afa9f51e9c649e3eb1" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 37537 } } peers { permanent_uuid: "cf43c478316e4b46a30860f95d7997b2" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 33731 } }
I20260807 15:11:30.444141 1344 tablet_replica.cc:439] T d77090f464e944beb3386ce5dd2d256d P 0cd27c0909644c38991b15cd2c5bc574: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0cd27c0909644c38991b15cd2c5bc574" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 34761 } } peers { permanent_uuid: "5a1ab2d4360447afa9f51e9c649e3eb1" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 37537 } } peers { permanent_uuid: "cf43c478316e4b46a30860f95d7997b2" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 33731 } } }
I20260807 15:11:30.444258 1340 ts_tablet_manager.cc:1428] T d77090f464e944beb3386ce5dd2d256d P 5a1ab2d4360447afa9f51e9c649e3eb1: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260807 15:11:30.444257 1344 tablet_replica.cc:442] T d77090f464e944beb3386ce5dd2d256d P 0cd27c0909644c38991b15cd2c5bc574: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 15:11:30.444476 1347 tablet_replica.cc:439] T d77090f464e944beb3386ce5dd2d256d P 5a1ab2d4360447afa9f51e9c649e3eb1: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0cd27c0909644c38991b15cd2c5bc574" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 34761 } } peers { permanent_uuid: "5a1ab2d4360447afa9f51e9c649e3eb1" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 37537 } } peers { permanent_uuid: "cf43c478316e4b46a30860f95d7997b2" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 33731 } } }
I20260807 15:11:30.444566 1347 tablet_replica.cc:442] T d77090f464e944beb3386ce5dd2d256d P 5a1ab2d4360447afa9f51e9c649e3eb1: This TxnStatusTablet replica's current role is: FOLLOWER
W20260807 15:11:30.456550 879 tablet.cc:2369] T 0267dcc371874c90a7649fd82dde29a7 P 0cd27c0909644c38991b15cd2c5bc574: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 15:11:30.461499 1344 raft_consensus.cc:482] T d77090f464e944beb3386ce5dd2d256d P 0cd27c0909644c38991b15cd2c5bc574 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 15:11:30.461694 1344 raft_consensus.cc:504] T d77090f464e944beb3386ce5dd2d256d P 0cd27c0909644c38991b15cd2c5bc574 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0cd27c0909644c38991b15cd2c5bc574" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 34761 } } peers { permanent_uuid: "5a1ab2d4360447afa9f51e9c649e3eb1" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 37537 } } peers { permanent_uuid: "cf43c478316e4b46a30860f95d7997b2" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 33731 } }
I20260807 15:11:30.462178 1344 leader_election.cc:288] T d77090f464e944beb3386ce5dd2d256d P 0cd27c0909644c38991b15cd2c5bc574 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5a1ab2d4360447afa9f51e9c649e3eb1 (127.29.194.2:37537), cf43c478316e4b46a30860f95d7997b2 (127.29.194.3:33731)
I20260807 15:11:30.466295 1329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:30.462814 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.194.1:41743 (local address 127.29.194.3:33731)
0807 15:11:30.462868 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:30.462885 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:30.462953 (+ 68us) server_negotiation.cc:407] Connection header received
0807 15:11:30.463064 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:30.463066 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:30.463079 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:30.463132 (+ 53us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:11:30.463318 (+ 186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:30.464165 (+ 847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:30.465498 (+ 1333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:30.465900 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:30.465967 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:30.465978 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0807 15:11:30.466165 (+ 187us) server_negotiation.cc:299] Negotiation successful
0807 15:11:30.466216 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260807 15:11:30.466316 1332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:30.462644 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.194.1:40369 (local address 127.29.194.2:37537)
0807 15:11:30.462711 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:30.462713 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:30.462728 (+ 15us) server_negotiation.cc:407] Connection header received
0807 15:11:30.462901 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:30.462902 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:30.462917 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:30.462970 (+ 53us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:11:30.463195 (+ 225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:30.464164 (+ 969us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:30.465519 (+ 1355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:30.465912 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:30.465990 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:30.465999 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0807 15:11:30.466201 (+ 202us) server_negotiation.cc:299] Negotiation successful
0807 15:11:30.466248 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260807 15:11:30.466338 1371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:30.462680 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.3:33731 (local address 127.29.194.1:41743)
0807 15:11:30.462862 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0807 15:11:30.462872 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 15:11:30.462961 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:11:30.463143 (+ 182us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:11:30.463144 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:11:30.463146 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:11:30.463228 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:30.463230 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:30.464366 (+ 1136us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:30.464368 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:30.465341 (+ 973us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:30.465346 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:30.465999 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:30.466001 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:30.466093 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:30.466097 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 15:11:30.466142 (+ 45us) client_negotiation.cc:240] Negotiation successful
0807 15:11:30.466188 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":160,"mutex_wait_us":45,"thread_start_us":114,"threads_started":1}
I20260807 15:11:30.466341 1370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:30.462455 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.2:37537 (local address 127.29.194.1:40369)
0807 15:11:30.462632 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0807 15:11:30.462645 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 15:11:30.462776 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:11:30.463003 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:11:30.463004 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:11:30.463007 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:11:30.463105 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:30.463108 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:30.464370 (+ 1262us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:30.464372 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:30.465342 (+ 970us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:30.465346 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:30.466027 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:30.466028 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:30.466124 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:30.466128 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 15:11:30.466185 (+ 57us) client_negotiation.cc:240] Negotiation successful
0807 15:11:30.466227 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154,"mutex_wait_us":104,"thread_start_us":99,"threads_started":1}
I20260807 15:11:30.466755 1229 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d77090f464e944beb3386ce5dd2d256d" candidate_uuid: "0cd27c0909644c38991b15cd2c5bc574" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf43c478316e4b46a30860f95d7997b2" is_pre_election: true
I20260807 15:11:30.466842 1229 raft_consensus.cc:2454] T d77090f464e944beb3386ce5dd2d256d P cf43c478316e4b46a30860f95d7997b2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0cd27c0909644c38991b15cd2c5bc574 in term 0.
I20260807 15:11:30.467060 717 leader_election.cc:302] T d77090f464e944beb3386ce5dd2d256d P 0cd27c0909644c38991b15cd2c5bc574 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0cd27c0909644c38991b15cd2c5bc574, cf43c478316e4b46a30860f95d7997b2; no voters:
I20260807 15:11:30.467211 1028 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d77090f464e944beb3386ce5dd2d256d" candidate_uuid: "0cd27c0909644c38991b15cd2c5bc574" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a1ab2d4360447afa9f51e9c649e3eb1" is_pre_election: true
I20260807 15:11:30.467279 1344 raft_consensus.cc:2781] T d77090f464e944beb3386ce5dd2d256d P 0cd27c0909644c38991b15cd2c5bc574 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 15:11:30.467339 1344 raft_consensus.cc:482] T d77090f464e944beb3386ce5dd2d256d P 0cd27c0909644c38991b15cd2c5bc574 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 15:11:30.467330 1028 raft_consensus.cc:2454] T d77090f464e944beb3386ce5dd2d256d P 5a1ab2d4360447afa9f51e9c649e3eb1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0cd27c0909644c38991b15cd2c5bc574 in term 0.
I20260807 15:11:30.467370 1344 raft_consensus.cc:3037] T d77090f464e944beb3386ce5dd2d256d P 0cd27c0909644c38991b15cd2c5bc574 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:11:30.468241 1344 raft_consensus.cc:504] T d77090f464e944beb3386ce5dd2d256d P 0cd27c0909644c38991b15cd2c5bc574 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0cd27c0909644c38991b15cd2c5bc574" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 34761 } } peers { permanent_uuid: "5a1ab2d4360447afa9f51e9c649e3eb1" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 37537 } } peers { permanent_uuid: "cf43c478316e4b46a30860f95d7997b2" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 33731 } }
I20260807 15:11:30.468376 1344 leader_election.cc:288] T d77090f464e944beb3386ce5dd2d256d P 0cd27c0909644c38991b15cd2c5bc574 [CANDIDATE]: Term 1 election: Requested vote from peers 5a1ab2d4360447afa9f51e9c649e3eb1 (127.29.194.2:37537), cf43c478316e4b46a30860f95d7997b2 (127.29.194.3:33731)
I20260807 15:11:30.468648 1028 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d77090f464e944beb3386ce5dd2d256d" candidate_uuid: "0cd27c0909644c38991b15cd2c5bc574" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a1ab2d4360447afa9f51e9c649e3eb1"
I20260807 15:11:30.468750 1028 raft_consensus.cc:3037] T d77090f464e944beb3386ce5dd2d256d P 5a1ab2d4360447afa9f51e9c649e3eb1 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:11:30.468729 1229 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d77090f464e944beb3386ce5dd2d256d" candidate_uuid: "0cd27c0909644c38991b15cd2c5bc574" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf43c478316e4b46a30860f95d7997b2"
I20260807 15:11:30.468801 1229 raft_consensus.cc:3037] T d77090f464e944beb3386ce5dd2d256d P cf43c478316e4b46a30860f95d7997b2 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:11:30.469650 1028 raft_consensus.cc:2454] T d77090f464e944beb3386ce5dd2d256d P 5a1ab2d4360447afa9f51e9c649e3eb1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0cd27c0909644c38991b15cd2c5bc574 in term 1.
I20260807 15:11:30.469650 1229 raft_consensus.cc:2454] T d77090f464e944beb3386ce5dd2d256d P cf43c478316e4b46a30860f95d7997b2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0cd27c0909644c38991b15cd2c5bc574 in term 1.
I20260807 15:11:30.469810 717 leader_election.cc:302] T d77090f464e944beb3386ce5dd2d256d P 0cd27c0909644c38991b15cd2c5bc574 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0cd27c0909644c38991b15cd2c5bc574, cf43c478316e4b46a30860f95d7997b2; no voters:
I20260807 15:11:30.469907 1344 raft_consensus.cc:2781] T d77090f464e944beb3386ce5dd2d256d P 0cd27c0909644c38991b15cd2c5bc574 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:11:30.470103 1344 raft_consensus.cc:686] T d77090f464e944beb3386ce5dd2d256d P 0cd27c0909644c38991b15cd2c5bc574 [term 1 LEADER]: Becoming Leader. State: Replica: 0cd27c0909644c38991b15cd2c5bc574, State: Running, Role: LEADER
I20260807 15:11:30.470219 1344 consensus_queue.cc:237] T d77090f464e944beb3386ce5dd2d256d P 0cd27c0909644c38991b15cd2c5bc574 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0cd27c0909644c38991b15cd2c5bc574" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 34761 } } peers { permanent_uuid: "5a1ab2d4360447afa9f51e9c649e3eb1" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 37537 } } peers { permanent_uuid: "cf43c478316e4b46a30860f95d7997b2" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 33731 } }
I20260807 15:11:30.471441 1374 tablet_replica.cc:439] T d77090f464e944beb3386ce5dd2d256d P 0cd27c0909644c38991b15cd2c5bc574: TxnStatusTablet state changed. Reason: New leader 0cd27c0909644c38991b15cd2c5bc574. Latest consensus state: current_term: 1 leader_uuid: "0cd27c0909644c38991b15cd2c5bc574" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0cd27c0909644c38991b15cd2c5bc574" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 34761 } } peers { permanent_uuid: "5a1ab2d4360447afa9f51e9c649e3eb1" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 37537 } } peers { permanent_uuid: "cf43c478316e4b46a30860f95d7997b2" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 33731 } } }
I20260807 15:11:30.471544 1374 tablet_replica.cc:442] T d77090f464e944beb3386ce5dd2d256d P 0cd27c0909644c38991b15cd2c5bc574: This TxnStatusTablet replica's current role is: LEADER
I20260807 15:11:30.471728 1377 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260807 15:11:30.471841 1377 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260807 15:11:30.471808 572 catalog_manager.cc:5495] T d77090f464e944beb3386ce5dd2d256d P 0cd27c0909644c38991b15cd2c5bc574 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0cd27c0909644c38991b15cd2c5bc574 (127.29.194.1). New cstate: current_term: 1 leader_uuid: "0cd27c0909644c38991b15cd2c5bc574" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0cd27c0909644c38991b15cd2c5bc574" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 34761 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5a1ab2d4360447afa9f51e9c649e3eb1" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 37537 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cf43c478316e4b46a30860f95d7997b2" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 33731 } health_report { overall_health: UNKNOWN } } }
I20260807 15:11:30.484647 1333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:30.481572 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53110 (local address 127.29.194.1:34761)
0807 15:11:30.481627 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:30.481630 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:30.481638 (+ 8us) server_negotiation.cc:407] Connection header received
0807 15:11:30.481676 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:30.481676 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:30.481693 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:30.481758 (+ 65us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 15:11:30.481946 (+ 188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:30.482432 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:30.483705 (+ 1273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:30.483906 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:30.483982 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:30.484273 (+ 291us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 15:11:30.484390 (+ 117us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 15:11:30.484465 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0807 15:11:30.484551 (+ 86us) server_negotiation.cc:299] Negotiation successful
0807 15:11:30.484577 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260807 15:11:30.484895 1365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:30.481417 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.1:34761 (local address 127.0.0.1:53110)
0807 15:11:30.481515 (+ 98us) negotiation.cc:107] Waiting for socket to connect
0807 15:11:30.481528 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 15:11:30.481580 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:11:30.481778 (+ 198us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:11:30.481779 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:11:30.481780 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 15:11:30.481840 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:30.481864 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:30.482546 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:30.482547 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:30.483558 (+ 1011us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:30.483565 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:30.484004 (+ 439us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:30.484005 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:30.484091 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:30.484122 (+ 31us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 15:11:30.484464 (+ 342us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 15:11:30.484472 (+ 8us) client_negotiation.cc:769] Sending connection context
0807 15:11:30.484524 (+ 52us) client_negotiation.cc:240] Negotiation successful
0807 15:11:30.484606 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":52}
I20260807 15:11:30.487304 1028 raft_consensus.cc:1261] T d77090f464e944beb3386ce5dd2d256d P 5a1ab2d4360447afa9f51e9c649e3eb1 [term 1 FOLLOWER]: Refusing update from remote peer 0cd27c0909644c38991b15cd2c5bc574: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:11:30.487295 1229 raft_consensus.cc:1261] T d77090f464e944beb3386ce5dd2d256d P cf43c478316e4b46a30860f95d7997b2 [term 1 FOLLOWER]: Refusing update from remote peer 0cd27c0909644c38991b15cd2c5bc574: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:11:30.487660 1344 consensus_queue.cc:1035] T d77090f464e944beb3386ce5dd2d256d P 0cd27c0909644c38991b15cd2c5bc574 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a1ab2d4360447afa9f51e9c649e3eb1" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 37537 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:11:30.487807 1374 consensus_queue.cc:1035] T d77090f464e944beb3386ce5dd2d256d P 0cd27c0909644c38991b15cd2c5bc574 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf43c478316e4b46a30860f95d7997b2" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 33731 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:11:30.489076 1346 tablet_replica.cc:439] T d77090f464e944beb3386ce5dd2d256d P cf43c478316e4b46a30860f95d7997b2: TxnStatusTablet state changed. Reason: New leader 0cd27c0909644c38991b15cd2c5bc574. Latest consensus state: current_term: 1 leader_uuid: "0cd27c0909644c38991b15cd2c5bc574" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0cd27c0909644c38991b15cd2c5bc574" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 34761 } } peers { permanent_uuid: "5a1ab2d4360447afa9f51e9c649e3eb1" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 37537 } } peers { permanent_uuid: "cf43c478316e4b46a30860f95d7997b2" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 33731 } } }
I20260807 15:11:30.489090 1347 tablet_replica.cc:439] T d77090f464e944beb3386ce5dd2d256d P 5a1ab2d4360447afa9f51e9c649e3eb1: TxnStatusTablet state changed. Reason: New leader 0cd27c0909644c38991b15cd2c5bc574. Latest consensus state: current_term: 1 leader_uuid: "0cd27c0909644c38991b15cd2c5bc574" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0cd27c0909644c38991b15cd2c5bc574" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 34761 } } peers { permanent_uuid: "5a1ab2d4360447afa9f51e9c649e3eb1" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 37537 } } peers { permanent_uuid: "cf43c478316e4b46a30860f95d7997b2" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 33731 } } }
I20260807 15:11:30.489176 1346 tablet_replica.cc:442] T d77090f464e944beb3386ce5dd2d256d P cf43c478316e4b46a30860f95d7997b2: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 15:11:30.489187 1347 tablet_replica.cc:442] T d77090f464e944beb3386ce5dd2d256d P 5a1ab2d4360447afa9f51e9c649e3eb1: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 15:11:30.490614 1344 tablet_replica.cc:439] T d77090f464e944beb3386ce5dd2d256d P 0cd27c0909644c38991b15cd2c5bc574: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0cd27c0909644c38991b15cd2c5bc574" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0cd27c0909644c38991b15cd2c5bc574" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 34761 } } peers { permanent_uuid: "5a1ab2d4360447afa9f51e9c649e3eb1" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 37537 } } peers { permanent_uuid: "cf43c478316e4b46a30860f95d7997b2" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 33731 } } }
I20260807 15:11:30.490718 1344 tablet_replica.cc:442] T d77090f464e944beb3386ce5dd2d256d P 0cd27c0909644c38991b15cd2c5bc574: This TxnStatusTablet replica's current role is: LEADER
I20260807 15:11:30.490722 1374 tablet_replica.cc:439] T d77090f464e944beb3386ce5dd2d256d P 0cd27c0909644c38991b15cd2c5bc574: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0cd27c0909644c38991b15cd2c5bc574" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0cd27c0909644c38991b15cd2c5bc574" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 34761 } } peers { permanent_uuid: "5a1ab2d4360447afa9f51e9c649e3eb1" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 37537 } } peers { permanent_uuid: "cf43c478316e4b46a30860f95d7997b2" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 33731 } } }
I20260807 15:11:30.491149 1374 tablet_replica.cc:442] T d77090f464e944beb3386ce5dd2d256d P 0cd27c0909644c38991b15cd2c5bc574: This TxnStatusTablet replica's current role is: LEADER
I20260807 15:11:30.491604 1383 mvcc.cc:205] Tried to move back new op lower bound from 7315929049032912896 to 7315929048968376320. Current Snapshot: MvccSnapshot[applied={T|T < 7315929049032912896}]
I20260807 15:11:30.491671 1347 tablet_replica.cc:439] T d77090f464e944beb3386ce5dd2d256d P 5a1ab2d4360447afa9f51e9c649e3eb1: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0cd27c0909644c38991b15cd2c5bc574" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0cd27c0909644c38991b15cd2c5bc574" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 34761 } } peers { permanent_uuid: "5a1ab2d4360447afa9f51e9c649e3eb1" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 37537 } } peers { permanent_uuid: "cf43c478316e4b46a30860f95d7997b2" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 33731 } } }
I20260807 15:11:30.491717 1346 tablet_replica.cc:439] T d77090f464e944beb3386ce5dd2d256d P cf43c478316e4b46a30860f95d7997b2: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0cd27c0909644c38991b15cd2c5bc574" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0cd27c0909644c38991b15cd2c5bc574" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 34761 } } peers { permanent_uuid: "5a1ab2d4360447afa9f51e9c649e3eb1" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 37537 } } peers { permanent_uuid: "cf43c478316e4b46a30860f95d7997b2" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 33731 } } }
I20260807 15:11:30.491762 1347 tablet_replica.cc:442] T d77090f464e944beb3386ce5dd2d256d P 5a1ab2d4360447afa9f51e9c649e3eb1: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 15:11:30.491786 1346 tablet_replica.cc:442] T d77090f464e944beb3386ce5dd2d256d P cf43c478316e4b46a30860f95d7997b2: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 15:11:30.492640 1384 mvcc.cc:205] Tried to move back new op lower bound from 7315929049032912896 to 7315929048968376320. Current Snapshot: MvccSnapshot[applied={T|T < 7315929049032912896 or (T in {7315929049032912896})}]
I20260807 15:11:30.496978 1332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:30.493825 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52224 (local address 127.29.194.2:37537)
0807 15:11:30.493868 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:30.493870 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:30.493897 (+ 27us) server_negotiation.cc:407] Connection header received
0807 15:11:30.493984 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:30.493984 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:30.493999 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:30.494055 (+ 56us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 15:11:30.494209 (+ 154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:30.494691 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:30.495916 (+ 1225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:30.496112 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:30.496178 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:30.496460 (+ 282us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 15:11:30.496655 (+ 195us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 15:11:30.496775 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0807 15:11:30.496846 (+ 71us) server_negotiation.cc:299] Negotiation successful
0807 15:11:30.496884 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260807 15:11:30.502291 1102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:30.499926 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.1:34761 (local address 127.29.194.2:49011)
0807 15:11:30.499997 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0807 15:11:30.500008 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 15:11:30.500047 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:11:30.500211 (+ 164us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:11:30.500212 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:11:30.500213 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 15:11:30.500280 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:30.500282 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:30.500887 (+ 605us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:30.500888 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:30.501552 (+ 664us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:30.501554 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:30.501873 (+ 319us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:30.501875 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:30.501943 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:30.501963 (+ 20us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 15:11:30.502133 (+ 170us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 15:11:30.502135 (+ 2us) client_negotiation.cc:769] Sending connection context
0807 15:11:30.502175 (+ 40us) client_negotiation.cc:240] Negotiation successful
0807 15:11:30.502221 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32}
I20260807 15:11:30.502283 1333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:30.500051 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.194.2:49011 (local address 127.29.194.1:34761)
0807 15:11:30.500117 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:30.500119 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:30.500126 (+ 7us) server_negotiation.cc:407] Connection header received
0807 15:11:30.500147 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:30.500148 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:30.500157 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:30.500201 (+ 44us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 15:11:30.500333 (+ 132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:30.500811 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:30.501634 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:30.501817 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:30.501869 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:30.502052 (+ 183us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 15:11:30.502084 (+ 32us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 15:11:30.502125 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0807 15:11:30.502176 (+ 51us) server_negotiation.cc:299] Negotiation successful
0807 15:11:30.502198 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260807 15:11:30.504915 1229 raft_consensus.cc:1261] T 0267dcc371874c90a7649fd82dde29a7 P cf43c478316e4b46a30860f95d7997b2 [term 1 FOLLOWER]: Refusing update from remote peer 5a1ab2d4360447afa9f51e9c649e3eb1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:11:30.505059 817 raft_consensus.cc:1261] T 0267dcc371874c90a7649fd82dde29a7 P 0cd27c0909644c38991b15cd2c5bc574 [term 1 FOLLOWER]: Refusing update from remote peer 5a1ab2d4360447afa9f51e9c649e3eb1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:11:30.505156 1347 consensus_queue.cc:1035] T 0267dcc371874c90a7649fd82dde29a7 P 5a1ab2d4360447afa9f51e9c649e3eb1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf43c478316e4b46a30860f95d7997b2" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 33731 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:11:30.505311 1350 consensus_queue.cc:1035] T 0267dcc371874c90a7649fd82dde29a7 P 5a1ab2d4360447afa9f51e9c649e3eb1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0cd27c0909644c38991b15cd2c5bc574" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 34761 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:11:30.507143 1382 mvcc.cc:205] Tried to move back new op lower bound from 7315929049106493440 to 7315929048643919872. Current Snapshot: MvccSnapshot[applied={T|T < 7315929049106493440}]
I20260807 15:11:30.515161 1405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:30.511786 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.2:37537 (local address 127.29.194.1:32871)
0807 15:11:30.511989 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0807 15:11:30.511998 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0807 15:11:30.512060 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:11:30.512210 (+ 150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:11:30.512210 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:11:30.512212 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 15:11:30.512284 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:30.512286 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:30.513218 (+ 932us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:30.513220 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:30.514210 (+ 990us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:30.514214 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:30.514761 (+ 547us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:30.514762 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:30.514820 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:30.514842 (+ 22us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 15:11:30.515010 (+ 168us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 15:11:30.515011 (+ 1us) client_negotiation.cc:769] Sending connection context
0807 15:11:30.515038 (+ 27us) client_negotiation.cc:240] Negotiation successful
0807 15:11:30.515078 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":89,"threads_started":1}
I20260807 15:11:30.515157 1332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:30.511845 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.194.1:32871 (local address 127.29.194.2:37537)
0807 15:11:30.511912 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:30.511914 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:30.512058 (+ 144us) server_negotiation.cc:407] Connection header received
0807 15:11:30.512106 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:30.512107 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:30.512118 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:30.512194 (+ 76us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 15:11:30.512343 (+ 149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:30.513145 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:30.514307 (+ 1162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:30.514616 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:30.514701 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:30.514924 (+ 223us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 15:11:30.514961 (+ 37us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 15:11:30.515005 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0807 15:11:30.515047 (+ 42us) server_negotiation.cc:299] Negotiation successful
0807 15:11:30.515072 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260807 15:11:30.525710 571 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40188:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 15:11:30.525871 571 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40188:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 15:11:30.526772 571 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 86993e10419047948fdd2437d6f0c4fc: Sending DeleteTablet for 3 replicas of tablet 0267dcc371874c90a7649fd82dde29a7
I20260807 15:11:30.527190 781 tablet_service.cc:1514] Processing DeleteTablet for tablet 0267dcc371874c90a7649fd82dde29a7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:11:30 UTC) from {username='kudu', principal='kudu/127.29.194.62@KRBTEST.COM'} at 127.0.0.1:53104
I20260807 15:11:30.527240 1205 tablet_service.cc:1514] Processing DeleteTablet for tablet 0267dcc371874c90a7649fd82dde29a7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:11:30 UTC) from {username='kudu', principal='kudu/127.29.194.62@KRBTEST.COM'} at 127.0.0.1:55238
I20260807 15:11:30.527302 998 tablet_service.cc:1514] Processing DeleteTablet for tablet 0267dcc371874c90a7649fd82dde29a7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:11:30 UTC) from {username='kudu', principal='kudu/127.29.194.62@KRBTEST.COM'} at 127.0.0.1:52208
I20260807 15:11:30.527612 1414 tablet_replica.cc:330] T 0267dcc371874c90a7649fd82dde29a7 P 5a1ab2d4360447afa9f51e9c649e3eb1: stopping tablet replica
I20260807 15:11:30.527733 1414 raft_consensus.cc:2229] T 0267dcc371874c90a7649fd82dde29a7 P 5a1ab2d4360447afa9f51e9c649e3eb1 [term 1 LEADER]: Raft consensus shutting down.
I20260807 15:11:30.527846 1414 raft_consensus.cc:2258] T 0267dcc371874c90a7649fd82dde29a7 P 5a1ab2d4360447afa9f51e9c649e3eb1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:11:30.528141 571 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40188:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 15:11:30.528321 571 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 15:11:30.528410 1414 ts_tablet_manager.cc:1905] T 0267dcc371874c90a7649fd82dde29a7 P 5a1ab2d4360447afa9f51e9c649e3eb1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:11:30.530272 781 tablet_service.cc:1467] Processing CreateTablet for tablet 7e260f9911e14850953fdb5913c61d18 (DEFAULT_TABLE table=test-table [id=527395e405c74d77a23a6ac1854ceb56]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:11:30.530458 781 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7e260f9911e14850953fdb5913c61d18. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:11:30.530567 1414 ts_tablet_manager.cc:1918] T 0267dcc371874c90a7649fd82dde29a7 P 5a1ab2d4360447afa9f51e9c649e3eb1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 15:11:30.530660 1414 log.cc:1196] T 0267dcc371874c90a7649fd82dde29a7 P 5a1ab2d4360447afa9f51e9c649e3eb1: Deleting WAL directory at /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115480851507-30472-0/minicluster-data/ts-1/wal/wals/0267dcc371874c90a7649fd82dde29a7
I20260807 15:11:30.530995 1414 ts_tablet_manager.cc:1939] T 0267dcc371874c90a7649fd82dde29a7 P 5a1ab2d4360447afa9f51e9c649e3eb1: Deleting consensus metadata
I20260807 15:11:30.532054 1337 tablet_bootstrap.cc:492] T 7e260f9911e14850953fdb5913c61d18 P 0cd27c0909644c38991b15cd2c5bc574: Bootstrap starting.
I20260807 15:11:30.532536 1337 tablet_bootstrap.cc:654] T 7e260f9911e14850953fdb5913c61d18 P 0cd27c0909644c38991b15cd2c5bc574: Neither blocks nor log segments found. Creating new log.
I20260807 15:11:30.533478 556 catalog_manager.cc:4841] TS 5a1ab2d4360447afa9f51e9c649e3eb1 (127.29.194.2:37537): tablet 0267dcc371874c90a7649fd82dde29a7 (table test-table [id=a2ab468749944c7292cfeff59d4f5d24]) successfully deleted
I20260807 15:11:30.533560 1411 tablet_replica.cc:330] T 0267dcc371874c90a7649fd82dde29a7 P 0cd27c0909644c38991b15cd2c5bc574: stopping tablet replica
I20260807 15:11:30.533695 1411 raft_consensus.cc:2229] T 0267dcc371874c90a7649fd82dde29a7 P 0cd27c0909644c38991b15cd2c5bc574 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 15:11:30.533757 1411 raft_consensus.cc:2258] T 0267dcc371874c90a7649fd82dde29a7 P 0cd27c0909644c38991b15cd2c5bc574 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:11:30.533803 998 tablet_service.cc:1467] Processing CreateTablet for tablet 7e260f9911e14850953fdb5913c61d18 (DEFAULT_TABLE table=test-table [id=527395e405c74d77a23a6ac1854ceb56]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:11:30.533929 998 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7e260f9911e14850953fdb5913c61d18. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:11:30.533880 1205 tablet_service.cc:1467] Processing CreateTablet for tablet 7e260f9911e14850953fdb5913c61d18 (DEFAULT_TABLE table=test-table [id=527395e405c74d77a23a6ac1854ceb56]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:11:30.534124 1205 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7e260f9911e14850953fdb5913c61d18. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:11:30.535521 1340 tablet_bootstrap.cc:492] T 7e260f9911e14850953fdb5913c61d18 P 5a1ab2d4360447afa9f51e9c649e3eb1: Bootstrap starting.
I20260807 15:11:30.536299 1340 tablet_bootstrap.cc:654] T 7e260f9911e14850953fdb5913c61d18 P 5a1ab2d4360447afa9f51e9c649e3eb1: Neither blocks nor log segments found. Creating new log.
I20260807 15:11:30.536521 1339 tablet_bootstrap.cc:492] T 7e260f9911e14850953fdb5913c61d18 P cf43c478316e4b46a30860f95d7997b2: Bootstrap starting.
I20260807 15:11:30.536881 1411 ts_tablet_manager.cc:1905] T 0267dcc371874c90a7649fd82dde29a7 P 0cd27c0909644c38991b15cd2c5bc574: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:11:30.537153 1339 tablet_bootstrap.cc:654] T 7e260f9911e14850953fdb5913c61d18 P cf43c478316e4b46a30860f95d7997b2: Neither blocks nor log segments found. Creating new log.
I20260807 15:11:30.537904 1337 tablet_bootstrap.cc:492] T 7e260f9911e14850953fdb5913c61d18 P 0cd27c0909644c38991b15cd2c5bc574: No bootstrap required, opened a new log
I20260807 15:11:30.537962 1337 ts_tablet_manager.cc:1397] T 7e260f9911e14850953fdb5913c61d18 P 0cd27c0909644c38991b15cd2c5bc574: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.000s
I20260807 15:11:30.538141 1337 raft_consensus.cc:348] T 7e260f9911e14850953fdb5913c61d18 P 0cd27c0909644c38991b15cd2c5bc574 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0cd27c0909644c38991b15cd2c5bc574" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 34761 } } peers { permanent_uuid: "5a1ab2d4360447afa9f51e9c649e3eb1" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 37537 } } peers { permanent_uuid: "cf43c478316e4b46a30860f95d7997b2" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 33731 } }
I20260807 15:11:30.538331 1337 raft_consensus.cc:374] T 7e260f9911e14850953fdb5913c61d18 P 0cd27c0909644c38991b15cd2c5bc574 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:11:30.538372 1337 raft_consensus.cc:729] T 7e260f9911e14850953fdb5913c61d18 P 0cd27c0909644c38991b15cd2c5bc574 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0cd27c0909644c38991b15cd2c5bc574, State: Initialized, Role: FOLLOWER
I20260807 15:11:30.538429 1337 consensus_queue.cc:260] T 7e260f9911e14850953fdb5913c61d18 P 0cd27c0909644c38991b15cd2c5bc574 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0cd27c0909644c38991b15cd2c5bc574" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 34761 } } peers { permanent_uuid: "5a1ab2d4360447afa9f51e9c649e3eb1" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 37537 } } peers { permanent_uuid: "cf43c478316e4b46a30860f95d7997b2" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 33731 } }
I20260807 15:11:30.538537 1337 ts_tablet_manager.cc:1428] T 7e260f9911e14850953fdb5913c61d18 P 0cd27c0909644c38991b15cd2c5bc574: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260807 15:11:30.539376 1411 ts_tablet_manager.cc:1918] T 0267dcc371874c90a7649fd82dde29a7 P 0cd27c0909644c38991b15cd2c5bc574: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 15:11:30.539459 1411 log.cc:1196] T 0267dcc371874c90a7649fd82dde29a7 P 0cd27c0909644c38991b15cd2c5bc574: Deleting WAL directory at /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115480851507-30472-0/minicluster-data/ts-0/wal/wals/0267dcc371874c90a7649fd82dde29a7
I20260807 15:11:30.540089 1411 ts_tablet_manager.cc:1939] T 0267dcc371874c90a7649fd82dde29a7 P 0cd27c0909644c38991b15cd2c5bc574: Deleting consensus metadata
I20260807 15:11:30.541788 555 catalog_manager.cc:4841] TS 0cd27c0909644c38991b15cd2c5bc574 (127.29.194.1:34761): tablet 0267dcc371874c90a7649fd82dde29a7 (table test-table [id=a2ab468749944c7292cfeff59d4f5d24]) successfully deleted
I20260807 15:11:30.541999 1413 tablet_replica.cc:330] T 0267dcc371874c90a7649fd82dde29a7 P cf43c478316e4b46a30860f95d7997b2: stopping tablet replica
I20260807 15:11:30.542196 1413 raft_consensus.cc:2229] T 0267dcc371874c90a7649fd82dde29a7 P cf43c478316e4b46a30860f95d7997b2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 15:11:30.542275 1413 raft_consensus.cc:2258] T 0267dcc371874c90a7649fd82dde29a7 P cf43c478316e4b46a30860f95d7997b2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:11:30.542279 1340 tablet_bootstrap.cc:492] T 7e260f9911e14850953fdb5913c61d18 P 5a1ab2d4360447afa9f51e9c649e3eb1: No bootstrap required, opened a new log
I20260807 15:11:30.542346 1340 ts_tablet_manager.cc:1397] T 7e260f9911e14850953fdb5913c61d18 P 5a1ab2d4360447afa9f51e9c649e3eb1: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.000s
I20260807 15:11:30.542562 1340 raft_consensus.cc:348] T 7e260f9911e14850953fdb5913c61d18 P 5a1ab2d4360447afa9f51e9c649e3eb1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0cd27c0909644c38991b15cd2c5bc574" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 34761 } } peers { permanent_uuid: "5a1ab2d4360447afa9f51e9c649e3eb1" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 37537 } } peers { permanent_uuid: "cf43c478316e4b46a30860f95d7997b2" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 33731 } }
I20260807 15:11:30.542644 1340 raft_consensus.cc:374] T 7e260f9911e14850953fdb5913c61d18 P 5a1ab2d4360447afa9f51e9c649e3eb1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:11:30.542662 1340 raft_consensus.cc:729] T 7e260f9911e14850953fdb5913c61d18 P 5a1ab2d4360447afa9f51e9c649e3eb1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a1ab2d4360447afa9f51e9c649e3eb1, State: Initialized, Role: FOLLOWER
I20260807 15:11:30.542716 1340 consensus_queue.cc:260] T 7e260f9911e14850953fdb5913c61d18 P 5a1ab2d4360447afa9f51e9c649e3eb1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0cd27c0909644c38991b15cd2c5bc574" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 34761 } } peers { permanent_uuid: "5a1ab2d4360447afa9f51e9c649e3eb1" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 37537 } } peers { permanent_uuid: "cf43c478316e4b46a30860f95d7997b2" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 33731 } }
I20260807 15:11:30.542819 1340 ts_tablet_manager.cc:1428] T 7e260f9911e14850953fdb5913c61d18 P 5a1ab2d4360447afa9f51e9c649e3eb1: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260807 15:11:30.542940 1413 ts_tablet_manager.cc:1905] T 0267dcc371874c90a7649fd82dde29a7 P cf43c478316e4b46a30860f95d7997b2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:11:30.545105 1413 ts_tablet_manager.cc:1918] T 0267dcc371874c90a7649fd82dde29a7 P cf43c478316e4b46a30860f95d7997b2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 15:11:30.545217 1413 log.cc:1196] T 0267dcc371874c90a7649fd82dde29a7 P cf43c478316e4b46a30860f95d7997b2: Deleting WAL directory at /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115480851507-30472-0/minicluster-data/ts-2/wal/wals/0267dcc371874c90a7649fd82dde29a7
I20260807 15:11:30.545779 1413 ts_tablet_manager.cc:1939] T 0267dcc371874c90a7649fd82dde29a7 P cf43c478316e4b46a30860f95d7997b2: Deleting consensus metadata
I20260807 15:11:30.545912 1339 tablet_bootstrap.cc:492] T 7e260f9911e14850953fdb5913c61d18 P cf43c478316e4b46a30860f95d7997b2: No bootstrap required, opened a new log
I20260807 15:11:30.545972 1339 ts_tablet_manager.cc:1397] T 7e260f9911e14850953fdb5913c61d18 P cf43c478316e4b46a30860f95d7997b2: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.001s
I20260807 15:11:30.546147 1339 raft_consensus.cc:348] T 7e260f9911e14850953fdb5913c61d18 P cf43c478316e4b46a30860f95d7997b2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0cd27c0909644c38991b15cd2c5bc574" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 34761 } } peers { permanent_uuid: "5a1ab2d4360447afa9f51e9c649e3eb1" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 37537 } } peers { permanent_uuid: "cf43c478316e4b46a30860f95d7997b2" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 33731 } }
I20260807 15:11:30.546238 1339 raft_consensus.cc:374] T 7e260f9911e14850953fdb5913c61d18 P cf43c478316e4b46a30860f95d7997b2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:11:30.546257 1339 raft_consensus.cc:729] T 7e260f9911e14850953fdb5913c61d18 P cf43c478316e4b46a30860f95d7997b2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf43c478316e4b46a30860f95d7997b2, State: Initialized, Role: FOLLOWER
I20260807 15:11:30.546298 1339 consensus_queue.cc:260] T 7e260f9911e14850953fdb5913c61d18 P cf43c478316e4b46a30860f95d7997b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0cd27c0909644c38991b15cd2c5bc574" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 34761 } } peers { permanent_uuid: "5a1ab2d4360447afa9f51e9c649e3eb1" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 37537 } } peers { permanent_uuid: "cf43c478316e4b46a30860f95d7997b2" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 33731 } }
I20260807 15:11:30.546394 1339 ts_tablet_manager.cc:1428] T 7e260f9911e14850953fdb5913c61d18 P cf43c478316e4b46a30860f95d7997b2: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260807 15:11:30.546495 556 catalog_manager.cc:4841] TS cf43c478316e4b46a30860f95d7997b2 (127.29.194.3:33731): tablet 0267dcc371874c90a7649fd82dde29a7 (table test-table [id=a2ab468749944c7292cfeff59d4f5d24]) successfully deleted
W20260807 15:11:30.559072 1299 tablet.cc:2369] T 7e260f9911e14850953fdb5913c61d18 P cf43c478316e4b46a30860f95d7997b2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 15:11:30.675947 1346 raft_consensus.cc:482] T 7e260f9911e14850953fdb5913c61d18 P cf43c478316e4b46a30860f95d7997b2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 15:11:30.676126 1346 raft_consensus.cc:504] T 7e260f9911e14850953fdb5913c61d18 P cf43c478316e4b46a30860f95d7997b2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0cd27c0909644c38991b15cd2c5bc574" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 34761 } } peers { permanent_uuid: "5a1ab2d4360447afa9f51e9c649e3eb1" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 37537 } } peers { permanent_uuid: "cf43c478316e4b46a30860f95d7997b2" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 33731 } }
I20260807 15:11:30.676602 1346 leader_election.cc:288] T 7e260f9911e14850953fdb5913c61d18 P cf43c478316e4b46a30860f95d7997b2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0cd27c0909644c38991b15cd2c5bc574 (127.29.194.1:34761), 5a1ab2d4360447afa9f51e9c649e3eb1 (127.29.194.2:37537)
I20260807 15:11:30.681123 1140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:30.676900 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.2:37537 (local address 127.29.194.3:46983)
0807 15:11:30.677030 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0807 15:11:30.677046 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 15:11:30.677102 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:11:30.677283 (+ 181us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:11:30.677284 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:11:30.677286 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:11:30.677404 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:30.677407 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:30.678342 (+ 935us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:30.678343 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:30.679671 (+ 1328us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:30.679675 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:30.680692 (+ 1017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:30.680694 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:30.680835 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:30.680843 (+ 8us) client_negotiation.cc:769] Sending connection context
0807 15:11:30.680953 (+ 110us) client_negotiation.cc:240] Negotiation successful
0807 15:11:30.681019 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":104,"mutex_wait_us":21}
I20260807 15:11:30.681167 1332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:30.677057 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.194.3:46983 (local address 127.29.194.2:37537)
0807 15:11:30.677123 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:30.677126 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:30.677139 (+ 13us) server_negotiation.cc:407] Connection header received
0807 15:11:30.677176 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:30.677177 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:30.677194 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:30.677256 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:11:30.677594 (+ 338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:30.678164 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:30.679814 (+ 1650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:30.680540 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:30.680645 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:30.680659 (+ 14us) server_negotiation.cc:1035] Waiting for connection context
0807 15:11:30.680992 (+ 333us) server_negotiation.cc:299] Negotiation successful
0807 15:11:30.681086 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260807 15:11:30.681200 1333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:30.677142 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.194.3:48669 (local address 127.29.194.1:34761)
0807 15:11:30.677221 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:30.677223 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:30.677232 (+ 9us) server_negotiation.cc:407] Connection header received
0807 15:11:30.677269 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:30.677270 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:30.677285 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:30.677346 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:11:30.678031 (+ 685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:30.678608 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:30.680041 (+ 1433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:30.680681 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:30.680751 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:30.680760 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0807 15:11:30.681058 (+ 298us) server_negotiation.cc:299] Negotiation successful
0807 15:11:30.681138 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260807 15:11:30.681175 1419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:30.676950 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.1:34761 (local address 127.29.194.3:48669)
0807 15:11:30.677126 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0807 15:11:30.677137 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 15:11:30.677219 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:11:30.677359 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:11:30.677359 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:11:30.677361 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:11:30.677911 (+ 550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:30.677915 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:30.678724 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:30.678725 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:30.679853 (+ 1128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:30.679856 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:30.680825 (+ 969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:30.680826 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:30.680901 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:30.680905 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 15:11:30.681004 (+ 99us) client_negotiation.cc:240] Negotiation successful
0807 15:11:30.681045 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":99,"threads_started":1}
I20260807 15:11:30.681674 1028 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7e260f9911e14850953fdb5913c61d18" candidate_uuid: "cf43c478316e4b46a30860f95d7997b2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a1ab2d4360447afa9f51e9c649e3eb1" is_pre_election: true
I20260807 15:11:30.681675 817 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7e260f9911e14850953fdb5913c61d18" candidate_uuid: "cf43c478316e4b46a30860f95d7997b2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0cd27c0909644c38991b15cd2c5bc574" is_pre_election: true
I20260807 15:11:30.681783 817 raft_consensus.cc:2454] T 7e260f9911e14850953fdb5913c61d18 P 0cd27c0909644c38991b15cd2c5bc574 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf43c478316e4b46a30860f95d7997b2 in term 0.
I20260807 15:11:30.681783 1028 raft_consensus.cc:2454] T 7e260f9911e14850953fdb5913c61d18 P 5a1ab2d4360447afa9f51e9c649e3eb1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf43c478316e4b46a30860f95d7997b2 in term 0.
I20260807 15:11:30.681990 1131 leader_election.cc:302] T 7e260f9911e14850953fdb5913c61d18 P cf43c478316e4b46a30860f95d7997b2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a1ab2d4360447afa9f51e9c649e3eb1, cf43c478316e4b46a30860f95d7997b2; no voters:
I20260807 15:11:30.682147 1346 raft_consensus.cc:2781] T 7e260f9911e14850953fdb5913c61d18 P cf43c478316e4b46a30860f95d7997b2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 15:11:30.682188 1346 raft_consensus.cc:482] T 7e260f9911e14850953fdb5913c61d18 P cf43c478316e4b46a30860f95d7997b2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 15:11:30.682204 1346 raft_consensus.cc:3037] T 7e260f9911e14850953fdb5913c61d18 P cf43c478316e4b46a30860f95d7997b2 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:11:30.683051 1346 raft_consensus.cc:504] T 7e260f9911e14850953fdb5913c61d18 P cf43c478316e4b46a30860f95d7997b2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0cd27c0909644c38991b15cd2c5bc574" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 34761 } } peers { permanent_uuid: "5a1ab2d4360447afa9f51e9c649e3eb1" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 37537 } } peers { permanent_uuid: "cf43c478316e4b46a30860f95d7997b2" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 33731 } }
I20260807 15:11:30.683219 1346 leader_election.cc:288] T 7e260f9911e14850953fdb5913c61d18 P cf43c478316e4b46a30860f95d7997b2 [CANDIDATE]: Term 1 election: Requested vote from peers 0cd27c0909644c38991b15cd2c5bc574 (127.29.194.1:34761), 5a1ab2d4360447afa9f51e9c649e3eb1 (127.29.194.2:37537)
I20260807 15:11:30.683393 1028 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7e260f9911e14850953fdb5913c61d18" candidate_uuid: "cf43c478316e4b46a30860f95d7997b2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a1ab2d4360447afa9f51e9c649e3eb1"
I20260807 15:11:30.683466 1028 raft_consensus.cc:3037] T 7e260f9911e14850953fdb5913c61d18 P 5a1ab2d4360447afa9f51e9c649e3eb1 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:11:30.683537 817 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7e260f9911e14850953fdb5913c61d18" candidate_uuid: "cf43c478316e4b46a30860f95d7997b2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0cd27c0909644c38991b15cd2c5bc574"
I20260807 15:11:30.683626 817 raft_consensus.cc:3037] T 7e260f9911e14850953fdb5913c61d18 P 0cd27c0909644c38991b15cd2c5bc574 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:11:30.684296 1028 raft_consensus.cc:2454] T 7e260f9911e14850953fdb5913c61d18 P 5a1ab2d4360447afa9f51e9c649e3eb1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf43c478316e4b46a30860f95d7997b2 in term 1.
I20260807 15:11:30.684459 817 raft_consensus.cc:2454] T 7e260f9911e14850953fdb5913c61d18 P 0cd27c0909644c38991b15cd2c5bc574 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf43c478316e4b46a30860f95d7997b2 in term 1.
I20260807 15:11:30.684617 1131 leader_election.cc:302] T 7e260f9911e14850953fdb5913c61d18 P cf43c478316e4b46a30860f95d7997b2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a1ab2d4360447afa9f51e9c649e3eb1, cf43c478316e4b46a30860f95d7997b2; no voters:
I20260807 15:11:30.684720 1346 raft_consensus.cc:2781] T 7e260f9911e14850953fdb5913c61d18 P cf43c478316e4b46a30860f95d7997b2 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:11:30.684906 1346 raft_consensus.cc:686] T 7e260f9911e14850953fdb5913c61d18 P cf43c478316e4b46a30860f95d7997b2 [term 1 LEADER]: Becoming Leader. State: Replica: cf43c478316e4b46a30860f95d7997b2, State: Running, Role: LEADER
I20260807 15:11:30.684975 1346 consensus_queue.cc:237] T 7e260f9911e14850953fdb5913c61d18 P cf43c478316e4b46a30860f95d7997b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0cd27c0909644c38991b15cd2c5bc574" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 34761 } } peers { permanent_uuid: "5a1ab2d4360447afa9f51e9c649e3eb1" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 37537 } } peers { permanent_uuid: "cf43c478316e4b46a30860f95d7997b2" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 33731 } }
I20260807 15:11:30.685880 572 catalog_manager.cc:5495] T 7e260f9911e14850953fdb5913c61d18 P cf43c478316e4b46a30860f95d7997b2 reported cstate change: term changed from 0 to 1, leader changed from <none> to cf43c478316e4b46a30860f95d7997b2 (127.29.194.3). New cstate: current_term: 1 leader_uuid: "cf43c478316e4b46a30860f95d7997b2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0cd27c0909644c38991b15cd2c5bc574" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 34761 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5a1ab2d4360447afa9f51e9c649e3eb1" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 37537 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cf43c478316e4b46a30860f95d7997b2" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 33731 } health_report { overall_health: HEALTHY } } }
I20260807 15:11:30.695946 1329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:30.693615 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55246 (local address 127.29.194.3:33731)
0807 15:11:30.693662 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:30.693663 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:30.693673 (+ 10us) server_negotiation.cc:407] Connection header received
0807 15:11:30.693759 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:30.693760 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:30.693786 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:30.693846 (+ 60us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 15:11:30.693955 (+ 109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:30.694460 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:30.695185 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:30.695379 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:30.695436 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:30.695596 (+ 160us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 15:11:30.695715 (+ 119us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 15:11:30.695785 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0807 15:11:30.695838 (+ 53us) server_negotiation.cc:299] Negotiation successful
0807 15:11:30.695863 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260807 15:11:30.702514 1308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:30.699732 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.1:34761 (local address 127.29.194.3:56569)
0807 15:11:30.699823 (+ 91us) negotiation.cc:107] Waiting for socket to connect
0807 15:11:30.699836 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 15:11:30.699890 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:11:30.700242 (+ 352us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:11:30.700244 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:11:30.700246 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 15:11:30.700359 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:30.700361 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:30.701070 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:30.701071 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:30.701734 (+ 663us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:30.701736 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:30.702074 (+ 338us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:30.702075 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:30.702143 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:30.702163 (+ 20us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 15:11:30.702334 (+ 171us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 15:11:30.702336 (+ 2us) client_negotiation.cc:769] Sending connection context
0807 15:11:30.702375 (+ 39us) client_negotiation.cc:240] Negotiation successful
0807 15:11:30.702436 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":25}
I20260807 15:11:30.702514 1333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:30.699888 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.194.3:56569 (local address 127.29.194.1:34761)
0807 15:11:30.700060 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:30.700065 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:30.700077 (+ 12us) server_negotiation.cc:407] Connection header received
0807 15:11:30.700114 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:30.700115 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:30.700130 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:30.700209 (+ 79us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 15:11:30.700428 (+ 219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:30.700986 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:30.701811 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:30.702012 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:30.702071 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:30.702242 (+ 171us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 15:11:30.702278 (+ 36us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 15:11:30.702319 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0807 15:11:30.702383 (+ 64us) server_negotiation.cc:299] Negotiation successful
0807 15:11:30.702407 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260807 15:11:30.705878 817 raft_consensus.cc:1261] T 7e260f9911e14850953fdb5913c61d18 P 0cd27c0909644c38991b15cd2c5bc574 [term 1 FOLLOWER]: Refusing update from remote peer cf43c478316e4b46a30860f95d7997b2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:11:30.706104 1028 raft_consensus.cc:1261] T 7e260f9911e14850953fdb5913c61d18 P 5a1ab2d4360447afa9f51e9c649e3eb1 [term 1 FOLLOWER]: Refusing update from remote peer cf43c478316e4b46a30860f95d7997b2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:11:30.706238 1421 consensus_queue.cc:1035] T 7e260f9911e14850953fdb5913c61d18 P cf43c478316e4b46a30860f95d7997b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0cd27c0909644c38991b15cd2c5bc574" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 34761 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:11:30.706388 1346 consensus_queue.cc:1035] T 7e260f9911e14850953fdb5913c61d18 P cf43c478316e4b46a30860f95d7997b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a1ab2d4360447afa9f51e9c649e3eb1" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 37537 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:11:30.715009 1405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:30.711928 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.3:33731 (local address 127.29.194.1:34203)
0807 15:11:30.711980 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0807 15:11:30.711993 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 15:11:30.712033 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:11:30.712254 (+ 221us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:11:30.712255 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:11:30.712256 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 15:11:30.712304 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:30.712306 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:30.713021 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:30.713022 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:30.713828 (+ 806us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:30.713831 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:30.714395 (+ 564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:30.714397 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:30.714506 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:30.714519 (+ 13us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 15:11:30.714792 (+ 273us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 15:11:30.714797 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 15:11:30.714871 (+ 74us) client_negotiation.cc:240] Negotiation successful
0807 15:11:30.714937 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":24}
I20260807 15:11:30.715025 1329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:30.712104 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.194.1:34203 (local address 127.29.194.3:33731)
0807 15:11:30.712156 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:30.712158 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:30.712165 (+ 7us) server_negotiation.cc:407] Connection header received
0807 15:11:30.712190 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:30.712190 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:30.712200 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:30.712251 (+ 51us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 15:11:30.712350 (+ 99us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:30.712923 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:30.713915 (+ 992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:30.714176 (+ 261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:30.714398 (+ 222us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:30.714608 (+ 210us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 15:11:30.714671 (+ 63us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 15:11:30.714735 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0807 15:11:30.714892 (+ 157us) server_negotiation.cc:299] Negotiation successful
0807 15:11:30.714929 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
W20260807 15:11:30.715559 1383 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260807 15:11:32.243906 572 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40188:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 15:11:32.244035 572 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40188:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 15:11:32.245301 572 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 86993e10419047948fdd2437d6f0c4fc: Sending DeleteTablet for 3 replicas of tablet 7e260f9911e14850953fdb5913c61d18
I20260807 15:11:32.245779 781 tablet_service.cc:1514] Processing DeleteTablet for tablet 7e260f9911e14850953fdb5913c61d18 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:11:32 UTC) from {username='kudu', principal='kudu/127.29.194.62@KRBTEST.COM'} at 127.0.0.1:53104
I20260807 15:11:32.245836 1205 tablet_service.cc:1514] Processing DeleteTablet for tablet 7e260f9911e14850953fdb5913c61d18 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:11:32 UTC) from {username='kudu', principal='kudu/127.29.194.62@KRBTEST.COM'} at 127.0.0.1:55238
I20260807 15:11:32.246023 998 tablet_service.cc:1514] Processing DeleteTablet for tablet 7e260f9911e14850953fdb5913c61d18 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:11:32 UTC) from {username='kudu', principal='kudu/127.29.194.62@KRBTEST.COM'} at 127.0.0.1:52208
I20260807 15:11:32.246387 1459 tablet_replica.cc:330] T 7e260f9911e14850953fdb5913c61d18 P 0cd27c0909644c38991b15cd2c5bc574: stopping tablet replica
I20260807 15:11:32.246441 1460 tablet_replica.cc:330] T 7e260f9911e14850953fdb5913c61d18 P cf43c478316e4b46a30860f95d7997b2: stopping tablet replica
I20260807 15:11:32.246464 1459 raft_consensus.cc:2229] T 7e260f9911e14850953fdb5913c61d18 P 0cd27c0909644c38991b15cd2c5bc574 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 15:11:32.246512 1459 raft_consensus.cc:2258] T 7e260f9911e14850953fdb5913c61d18 P 0cd27c0909644c38991b15cd2c5bc574 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:11:32.246552 1460 raft_consensus.cc:2229] T 7e260f9911e14850953fdb5913c61d18 P cf43c478316e4b46a30860f95d7997b2 [term 1 LEADER]: Raft consensus shutting down.
I20260807 15:11:32.246661 1460 raft_consensus.cc:2258] T 7e260f9911e14850953fdb5913c61d18 P cf43c478316e4b46a30860f95d7997b2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:11:32.246776 1462 tablet_replica.cc:330] T 7e260f9911e14850953fdb5913c61d18 P 5a1ab2d4360447afa9f51e9c649e3eb1: stopping tablet replica
I20260807 15:11:32.246837 1462 raft_consensus.cc:2229] T 7e260f9911e14850953fdb5913c61d18 P 5a1ab2d4360447afa9f51e9c649e3eb1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 15:11:32.246830 1459 ts_tablet_manager.cc:1905] T 7e260f9911e14850953fdb5913c61d18 P 0cd27c0909644c38991b15cd2c5bc574: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:11:32.246882 1462 raft_consensus.cc:2258] T 7e260f9911e14850953fdb5913c61d18 P 5a1ab2d4360447afa9f51e9c649e3eb1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:11:32.247152 1462 ts_tablet_manager.cc:1905] T 7e260f9911e14850953fdb5913c61d18 P 5a1ab2d4360447afa9f51e9c649e3eb1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:11:32.248484 1459 ts_tablet_manager.cc:1918] T 7e260f9911e14850953fdb5913c61d18 P 0cd27c0909644c38991b15cd2c5bc574: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260807 15:11:32.248591 1459 log.cc:1196] T 7e260f9911e14850953fdb5913c61d18 P 0cd27c0909644c38991b15cd2c5bc574: Deleting WAL directory at /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115480851507-30472-0/minicluster-data/ts-0/wal/wals/7e260f9911e14850953fdb5913c61d18
I20260807 15:11:32.248837 1459 ts_tablet_manager.cc:1939] T 7e260f9911e14850953fdb5913c61d18 P 0cd27c0909644c38991b15cd2c5bc574: Deleting consensus metadata
I20260807 15:11:32.248873 1462 ts_tablet_manager.cc:1918] T 7e260f9911e14850953fdb5913c61d18 P 5a1ab2d4360447afa9f51e9c649e3eb1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260807 15:11:32.249034 1462 log.cc:1196] T 7e260f9911e14850953fdb5913c61d18 P 5a1ab2d4360447afa9f51e9c649e3eb1: Deleting WAL directory at /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115480851507-30472-0/minicluster-data/ts-1/wal/wals/7e260f9911e14850953fdb5913c61d18
I20260807 15:11:32.249297 1460 ts_tablet_manager.cc:1905] T 7e260f9911e14850953fdb5913c61d18 P cf43c478316e4b46a30860f95d7997b2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:11:32.249318 1462 ts_tablet_manager.cc:1939] T 7e260f9911e14850953fdb5913c61d18 P 5a1ab2d4360447afa9f51e9c649e3eb1: Deleting consensus metadata
I20260807 15:11:32.249323 555 catalog_manager.cc:4841] TS 0cd27c0909644c38991b15cd2c5bc574 (127.29.194.1:34761): tablet 7e260f9911e14850953fdb5913c61d18 (table test-table [id=527395e405c74d77a23a6ac1854ceb56]) successfully deleted
I20260807 15:11:32.249668 556 catalog_manager.cc:4841] TS 5a1ab2d4360447afa9f51e9c649e3eb1 (127.29.194.2:37537): tablet 7e260f9911e14850953fdb5913c61d18 (table test-table [id=527395e405c74d77a23a6ac1854ceb56]) successfully deleted
I20260807 15:11:32.251039 1460 ts_tablet_manager.cc:1918] T 7e260f9911e14850953fdb5913c61d18 P cf43c478316e4b46a30860f95d7997b2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260807 15:11:32.251092 1460 log.cc:1196] T 7e260f9911e14850953fdb5913c61d18 P cf43c478316e4b46a30860f95d7997b2: Deleting WAL directory at /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115480851507-30472-0/minicluster-data/ts-2/wal/wals/7e260f9911e14850953fdb5913c61d18
I20260807 15:11:32.251327 1460 ts_tablet_manager.cc:1939] T 7e260f9911e14850953fdb5913c61d18 P cf43c478316e4b46a30860f95d7997b2: Deleting consensus metadata
I20260807 15:11:32.252033 556 catalog_manager.cc:4841] TS cf43c478316e4b46a30860f95d7997b2 (127.29.194.3:33731): tablet 7e260f9911e14850953fdb5913c61d18 (table test-table [id=527395e405c74d77a23a6ac1854ceb56]) successfully deleted
Aug 07 15:11:32 dist-test-slave-42z9 krb5kdc[502](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115492, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 07 15:11:32 dist-test-slave-42z9 krb5kdc[502](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115492, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.29.194.62@KRBTEST.COM
I20260807 15:11:32.278187 1472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:32.266430 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40200 (local address 127.29.194.62:43411)
0807 15:11:32.266584 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:32.266586 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:32.266595 (+ 9us) server_negotiation.cc:407] Connection header received
0807 15:11:32.266623 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:32.266623 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:32.266634 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:32.266723 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:11:32.267319 (+ 596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:32.267874 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:32.268669 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:32.268872 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:32.268926 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:32.274873 (+ 5947us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:11:32.274875 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:11:32.274875 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:11:32.274884 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:11:32.276394 (+ 1510us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:32.276878 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:32.276879 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:32.276880 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:32.276938 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:32.277201 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:32.277201 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:32.277201 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:32.277326 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:11:32.277411 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0807 15:11:32.277967 (+ 556us) server_negotiation.cc:299] Negotiation successful
0807 15:11:32.278070 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":128,"thread_start_us":90,"threads_started":1}
I20260807 15:11:32.280021 572 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40200:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 15:11:32.280207 572 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 15:11:32.281945 998 tablet_service.cc:1467] Processing CreateTablet for tablet 3e6246af76a443c0ab45f5d0ee94057c (DEFAULT_TABLE table=test-table [id=2267f5ef59204ac7808f0ee39c7da91e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:11:32.281945 781 tablet_service.cc:1467] Processing CreateTablet for tablet 3e6246af76a443c0ab45f5d0ee94057c (DEFAULT_TABLE table=test-table [id=2267f5ef59204ac7808f0ee39c7da91e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:11:32.282032 1205 tablet_service.cc:1467] Processing CreateTablet for tablet 3e6246af76a443c0ab45f5d0ee94057c (DEFAULT_TABLE table=test-table [id=2267f5ef59204ac7808f0ee39c7da91e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:11:32.282106 781 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3e6246af76a443c0ab45f5d0ee94057c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:11:32.282106 998 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3e6246af76a443c0ab45f5d0ee94057c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:11:32.282153 1205 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3e6246af76a443c0ab45f5d0ee94057c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:11:32.283782 1473 tablet_bootstrap.cc:492] T 3e6246af76a443c0ab45f5d0ee94057c P cf43c478316e4b46a30860f95d7997b2: Bootstrap starting.
I20260807 15:11:32.284211 1473 tablet_bootstrap.cc:654] T 3e6246af76a443c0ab45f5d0ee94057c P cf43c478316e4b46a30860f95d7997b2: Neither blocks nor log segments found. Creating new log.
I20260807 15:11:32.284533 1475 tablet_bootstrap.cc:492] T 3e6246af76a443c0ab45f5d0ee94057c P 0cd27c0909644c38991b15cd2c5bc574: Bootstrap starting.
I20260807 15:11:32.284951 1473 tablet_bootstrap.cc:492] T 3e6246af76a443c0ab45f5d0ee94057c P cf43c478316e4b46a30860f95d7997b2: No bootstrap required, opened a new log
I20260807 15:11:32.284987 1475 tablet_bootstrap.cc:654] T 3e6246af76a443c0ab45f5d0ee94057c P 0cd27c0909644c38991b15cd2c5bc574: Neither blocks nor log segments found. Creating new log.
I20260807 15:11:32.285020 1473 ts_tablet_manager.cc:1397] T 3e6246af76a443c0ab45f5d0ee94057c P cf43c478316e4b46a30860f95d7997b2: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260807 15:11:32.285142 1473 raft_consensus.cc:348] T 3e6246af76a443c0ab45f5d0ee94057c P cf43c478316e4b46a30860f95d7997b2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0cd27c0909644c38991b15cd2c5bc574" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 34761 } } peers { permanent_uuid: "5a1ab2d4360447afa9f51e9c649e3eb1" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 37537 } } peers { permanent_uuid: "cf43c478316e4b46a30860f95d7997b2" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 33731 } }
I20260807 15:11:32.285329 1473 raft_consensus.cc:374] T 3e6246af76a443c0ab45f5d0ee94057c P cf43c478316e4b46a30860f95d7997b2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:11:32.285357 1474 tablet_bootstrap.cc:492] T 3e6246af76a443c0ab45f5d0ee94057c P 5a1ab2d4360447afa9f51e9c649e3eb1: Bootstrap starting.
I20260807 15:11:32.285378 1473 raft_consensus.cc:729] T 3e6246af76a443c0ab45f5d0ee94057c P cf43c478316e4b46a30860f95d7997b2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf43c478316e4b46a30860f95d7997b2, State: Initialized, Role: FOLLOWER
I20260807 15:11:32.285430 1473 consensus_queue.cc:260] T 3e6246af76a443c0ab45f5d0ee94057c P cf43c478316e4b46a30860f95d7997b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0cd27c0909644c38991b15cd2c5bc574" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 34761 } } peers { permanent_uuid: "5a1ab2d4360447afa9f51e9c649e3eb1" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 37537 } } peers { permanent_uuid: "cf43c478316e4b46a30860f95d7997b2" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 33731 } }
I20260807 15:11:32.285667 1473 ts_tablet_manager.cc:1428] T 3e6246af76a443c0ab45f5d0ee94057c P cf43c478316e4b46a30860f95d7997b2: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260807 15:11:32.285898 1474 tablet_bootstrap.cc:654] T 3e6246af76a443c0ab45f5d0ee94057c P 5a1ab2d4360447afa9f51e9c649e3eb1: Neither blocks nor log segments found. Creating new log.
I20260807 15:11:32.286309 1475 tablet_bootstrap.cc:492] T 3e6246af76a443c0ab45f5d0ee94057c P 0cd27c0909644c38991b15cd2c5bc574: No bootstrap required, opened a new log
I20260807 15:11:32.286401 1475 ts_tablet_manager.cc:1397] T 3e6246af76a443c0ab45f5d0ee94057c P 0cd27c0909644c38991b15cd2c5bc574: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 15:11:32.286583 1474 tablet_bootstrap.cc:492] T 3e6246af76a443c0ab45f5d0ee94057c P 5a1ab2d4360447afa9f51e9c649e3eb1: No bootstrap required, opened a new log
I20260807 15:11:32.286547 1475 raft_consensus.cc:348] T 3e6246af76a443c0ab45f5d0ee94057c P 0cd27c0909644c38991b15cd2c5bc574 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0cd27c0909644c38991b15cd2c5bc574" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 34761 } } peers { permanent_uuid: "5a1ab2d4360447afa9f51e9c649e3eb1" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 37537 } } peers { permanent_uuid: "cf43c478316e4b46a30860f95d7997b2" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 33731 } }
I20260807 15:11:32.286624 1475 raft_consensus.cc:374] T 3e6246af76a443c0ab45f5d0ee94057c P 0cd27c0909644c38991b15cd2c5bc574 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:11:32.286633 1474 ts_tablet_manager.cc:1397] T 3e6246af76a443c0ab45f5d0ee94057c P 5a1ab2d4360447afa9f51e9c649e3eb1: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260807 15:11:32.286654 1475 raft_consensus.cc:729] T 3e6246af76a443c0ab45f5d0ee94057c P 0cd27c0909644c38991b15cd2c5bc574 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0cd27c0909644c38991b15cd2c5bc574, State: Initialized, Role: FOLLOWER
I20260807 15:11:32.286727 1475 consensus_queue.cc:260] T 3e6246af76a443c0ab45f5d0ee94057c P 0cd27c0909644c38991b15cd2c5bc574 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0cd27c0909644c38991b15cd2c5bc574" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 34761 } } peers { permanent_uuid: "5a1ab2d4360447afa9f51e9c649e3eb1" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 37537 } } peers { permanent_uuid: "cf43c478316e4b46a30860f95d7997b2" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 33731 } }
I20260807 15:11:32.286751 1474 raft_consensus.cc:348] T 3e6246af76a443c0ab45f5d0ee94057c P 5a1ab2d4360447afa9f51e9c649e3eb1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0cd27c0909644c38991b15cd2c5bc574" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 34761 } } peers { permanent_uuid: "5a1ab2d4360447afa9f51e9c649e3eb1" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 37537 } } peers { permanent_uuid: "cf43c478316e4b46a30860f95d7997b2" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 33731 } }
I20260807 15:11:32.286845 1475 ts_tablet_manager.cc:1428] T 3e6246af76a443c0ab45f5d0ee94057c P 0cd27c0909644c38991b15cd2c5bc574: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260807 15:11:32.286888 1474 raft_consensus.cc:374] T 3e6246af76a443c0ab45f5d0ee94057c P 5a1ab2d4360447afa9f51e9c649e3eb1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:11:32.286914 1474 raft_consensus.cc:729] T 3e6246af76a443c0ab45f5d0ee94057c P 5a1ab2d4360447afa9f51e9c649e3eb1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a1ab2d4360447afa9f51e9c649e3eb1, State: Initialized, Role: FOLLOWER
I20260807 15:11:32.286985 1474 consensus_queue.cc:260] T 3e6246af76a443c0ab45f5d0ee94057c P 5a1ab2d4360447afa9f51e9c649e3eb1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0cd27c0909644c38991b15cd2c5bc574" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 34761 } } peers { permanent_uuid: "5a1ab2d4360447afa9f51e9c649e3eb1" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 37537 } } peers { permanent_uuid: "cf43c478316e4b46a30860f95d7997b2" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 33731 } }
I20260807 15:11:32.287165 1474 ts_tablet_manager.cc:1428] T 3e6246af76a443c0ab45f5d0ee94057c P 5a1ab2d4360447afa9f51e9c649e3eb1: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260807 15:11:32.395555 1481 raft_consensus.cc:482] T 3e6246af76a443c0ab45f5d0ee94057c P 5a1ab2d4360447afa9f51e9c649e3eb1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 15:11:32.395678 1481 raft_consensus.cc:504] T 3e6246af76a443c0ab45f5d0ee94057c P 5a1ab2d4360447afa9f51e9c649e3eb1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0cd27c0909644c38991b15cd2c5bc574" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 34761 } } peers { permanent_uuid: "5a1ab2d4360447afa9f51e9c649e3eb1" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 37537 } } peers { permanent_uuid: "cf43c478316e4b46a30860f95d7997b2" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 33731 } }
I20260807 15:11:32.395879 1481 leader_election.cc:288] T 3e6246af76a443c0ab45f5d0ee94057c P 5a1ab2d4360447afa9f51e9c649e3eb1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0cd27c0909644c38991b15cd2c5bc574 (127.29.194.1:34761), cf43c478316e4b46a30860f95d7997b2 (127.29.194.3:33731)
I20260807 15:11:32.396258 817 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3e6246af76a443c0ab45f5d0ee94057c" candidate_uuid: "5a1ab2d4360447afa9f51e9c649e3eb1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0cd27c0909644c38991b15cd2c5bc574" is_pre_election: true
I20260807 15:11:32.396375 817 raft_consensus.cc:2454] T 3e6246af76a443c0ab45f5d0ee94057c P 0cd27c0909644c38991b15cd2c5bc574 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a1ab2d4360447afa9f51e9c649e3eb1 in term 0.
I20260807 15:11:32.396394 1229 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3e6246af76a443c0ab45f5d0ee94057c" candidate_uuid: "5a1ab2d4360447afa9f51e9c649e3eb1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf43c478316e4b46a30860f95d7997b2" is_pre_election: true
I20260807 15:11:32.396610 923 leader_election.cc:302] T 3e6246af76a443c0ab45f5d0ee94057c P 5a1ab2d4360447afa9f51e9c649e3eb1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0cd27c0909644c38991b15cd2c5bc574, 5a1ab2d4360447afa9f51e9c649e3eb1; no voters:
I20260807 15:11:32.396601 1229 raft_consensus.cc:2454] T 3e6246af76a443c0ab45f5d0ee94057c P cf43c478316e4b46a30860f95d7997b2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a1ab2d4360447afa9f51e9c649e3eb1 in term 0.
I20260807 15:11:32.396754 1481 raft_consensus.cc:2781] T 3e6246af76a443c0ab45f5d0ee94057c P 5a1ab2d4360447afa9f51e9c649e3eb1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 15:11:32.396802 1481 raft_consensus.cc:482] T 3e6246af76a443c0ab45f5d0ee94057c P 5a1ab2d4360447afa9f51e9c649e3eb1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 15:11:32.396824 1481 raft_consensus.cc:3037] T 3e6246af76a443c0ab45f5d0ee94057c P 5a1ab2d4360447afa9f51e9c649e3eb1 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:11:32.397723 1481 raft_consensus.cc:504] T 3e6246af76a443c0ab45f5d0ee94057c P 5a1ab2d4360447afa9f51e9c649e3eb1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0cd27c0909644c38991b15cd2c5bc574" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 34761 } } peers { permanent_uuid: "5a1ab2d4360447afa9f51e9c649e3eb1" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 37537 } } peers { permanent_uuid: "cf43c478316e4b46a30860f95d7997b2" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 33731 } }
I20260807 15:11:32.397883 1481 leader_election.cc:288] T 3e6246af76a443c0ab45f5d0ee94057c P 5a1ab2d4360447afa9f51e9c649e3eb1 [CANDIDATE]: Term 1 election: Requested vote from peers 0cd27c0909644c38991b15cd2c5bc574 (127.29.194.1:34761), cf43c478316e4b46a30860f95d7997b2 (127.29.194.3:33731)
I20260807 15:11:32.398061 817 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3e6246af76a443c0ab45f5d0ee94057c" candidate_uuid: "5a1ab2d4360447afa9f51e9c649e3eb1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0cd27c0909644c38991b15cd2c5bc574"
I20260807 15:11:32.398146 817 raft_consensus.cc:3037] T 3e6246af76a443c0ab45f5d0ee94057c P 0cd27c0909644c38991b15cd2c5bc574 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:11:32.398214 1229 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3e6246af76a443c0ab45f5d0ee94057c" candidate_uuid: "5a1ab2d4360447afa9f51e9c649e3eb1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf43c478316e4b46a30860f95d7997b2"
I20260807 15:11:32.398351 1229 raft_consensus.cc:3037] T 3e6246af76a443c0ab45f5d0ee94057c P cf43c478316e4b46a30860f95d7997b2 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:11:32.398916 817 raft_consensus.cc:2454] T 3e6246af76a443c0ab45f5d0ee94057c P 0cd27c0909644c38991b15cd2c5bc574 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a1ab2d4360447afa9f51e9c649e3eb1 in term 1.
I20260807 15:11:32.399206 923 leader_election.cc:302] T 3e6246af76a443c0ab45f5d0ee94057c P 5a1ab2d4360447afa9f51e9c649e3eb1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0cd27c0909644c38991b15cd2c5bc574, 5a1ab2d4360447afa9f51e9c649e3eb1; no voters:
I20260807 15:11:32.399349 1481 raft_consensus.cc:2781] T 3e6246af76a443c0ab45f5d0ee94057c P 5a1ab2d4360447afa9f51e9c649e3eb1 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:11:32.399435 1229 raft_consensus.cc:2454] T 3e6246af76a443c0ab45f5d0ee94057c P cf43c478316e4b46a30860f95d7997b2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a1ab2d4360447afa9f51e9c649e3eb1 in term 1.
I20260807 15:11:32.399626 1481 raft_consensus.cc:686] T 3e6246af76a443c0ab45f5d0ee94057c P 5a1ab2d4360447afa9f51e9c649e3eb1 [term 1 LEADER]: Becoming Leader. State: Replica: 5a1ab2d4360447afa9f51e9c649e3eb1, State: Running, Role: LEADER
I20260807 15:11:32.399794 1481 consensus_queue.cc:237] T 3e6246af76a443c0ab45f5d0ee94057c P 5a1ab2d4360447afa9f51e9c649e3eb1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0cd27c0909644c38991b15cd2c5bc574" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 34761 } } peers { permanent_uuid: "5a1ab2d4360447afa9f51e9c649e3eb1" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 37537 } } peers { permanent_uuid: "cf43c478316e4b46a30860f95d7997b2" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 33731 } }
I20260807 15:11:32.400640 571 catalog_manager.cc:5495] T 3e6246af76a443c0ab45f5d0ee94057c P 5a1ab2d4360447afa9f51e9c649e3eb1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5a1ab2d4360447afa9f51e9c649e3eb1 (127.29.194.2). New cstate: current_term: 1 leader_uuid: "5a1ab2d4360447afa9f51e9c649e3eb1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0cd27c0909644c38991b15cd2c5bc574" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 34761 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5a1ab2d4360447afa9f51e9c649e3eb1" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 37537 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cf43c478316e4b46a30860f95d7997b2" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 33731 } health_report { overall_health: UNKNOWN } } }
I20260807 15:11:32.438043 1486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:32.434977 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52240 (local address 127.29.194.2:37537)
0807 15:11:32.435191 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:32.435193 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:32.435201 (+ 8us) server_negotiation.cc:407] Connection header received
0807 15:11:32.435231 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:32.435232 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:32.435245 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:32.435326 (+ 81us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 15:11:32.435640 (+ 314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:32.436188 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:32.437180 (+ 992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:32.437403 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:32.437516 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:32.437709 (+ 193us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 15:11:32.437752 (+ 43us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 15:11:32.437804 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0807 15:11:32.437909 (+ 105us) server_negotiation.cc:299] Negotiation successful
0807 15:11:32.437936 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":183,"thread_start_us":147,"threads_started":1}
I20260807 15:11:32.441385 1229 raft_consensus.cc:1261] T 3e6246af76a443c0ab45f5d0ee94057c P cf43c478316e4b46a30860f95d7997b2 [term 1 FOLLOWER]: Refusing update from remote peer 5a1ab2d4360447afa9f51e9c649e3eb1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:11:32.441406 817 raft_consensus.cc:1261] T 3e6246af76a443c0ab45f5d0ee94057c P 0cd27c0909644c38991b15cd2c5bc574 [term 1 FOLLOWER]: Refusing update from remote peer 5a1ab2d4360447afa9f51e9c649e3eb1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:11:32.441735 1481 consensus_queue.cc:1035] T 3e6246af76a443c0ab45f5d0ee94057c P 5a1ab2d4360447afa9f51e9c649e3eb1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf43c478316e4b46a30860f95d7997b2" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 33731 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:11:32.441887 1483 consensus_queue.cc:1035] T 3e6246af76a443c0ab45f5d0ee94057c P 5a1ab2d4360447afa9f51e9c649e3eb1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0cd27c0909644c38991b15cd2c5bc574" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 34761 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:11:32.454221 571 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40200:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 15:11:32.454376 571 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40200:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 15:11:32.455224 571 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 86993e10419047948fdd2437d6f0c4fc: Sending DeleteTablet for 3 replicas of tablet 3e6246af76a443c0ab45f5d0ee94057c
I20260807 15:11:32.455647 1205 tablet_service.cc:1514] Processing DeleteTablet for tablet 3e6246af76a443c0ab45f5d0ee94057c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:11:32 UTC) from {username='kudu', principal='kudu/127.29.194.62@KRBTEST.COM'} at 127.0.0.1:55238
I20260807 15:11:32.455693 998 tablet_service.cc:1514] Processing DeleteTablet for tablet 3e6246af76a443c0ab45f5d0ee94057c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:11:32 UTC) from {username='kudu', principal='kudu/127.29.194.62@KRBTEST.COM'} at 127.0.0.1:52208
I20260807 15:11:32.455797 1460 tablet_replica.cc:330] T 3e6246af76a443c0ab45f5d0ee94057c P cf43c478316e4b46a30860f95d7997b2: stopping tablet replica
I20260807 15:11:32.455819 1462 tablet_replica.cc:330] T 3e6246af76a443c0ab45f5d0ee94057c P 5a1ab2d4360447afa9f51e9c649e3eb1: stopping tablet replica
I20260807 15:11:32.455874 1460 raft_consensus.cc:2229] T 3e6246af76a443c0ab45f5d0ee94057c P cf43c478316e4b46a30860f95d7997b2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 15:11:32.455895 1462 raft_consensus.cc:2229] T 3e6246af76a443c0ab45f5d0ee94057c P 5a1ab2d4360447afa9f51e9c649e3eb1 [term 1 LEADER]: Raft consensus shutting down.
I20260807 15:11:32.455932 1460 raft_consensus.cc:2258] T 3e6246af76a443c0ab45f5d0ee94057c P cf43c478316e4b46a30860f95d7997b2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:11:32.455977 1462 raft_consensus.cc:2258] T 3e6246af76a443c0ab45f5d0ee94057c P 5a1ab2d4360447afa9f51e9c649e3eb1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:11:32.455914 571 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40200:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 15:11:32.456040 781 tablet_service.cc:1514] Processing DeleteTablet for tablet 3e6246af76a443c0ab45f5d0ee94057c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:11:32 UTC) from {username='kudu', principal='kudu/127.29.194.62@KRBTEST.COM'} at 127.0.0.1:53104
W20260807 15:11:32.456110 571 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 15:11:32.456156 1459 tablet_replica.cc:330] T 3e6246af76a443c0ab45f5d0ee94057c P 0cd27c0909644c38991b15cd2c5bc574: stopping tablet replica
I20260807 15:11:32.456236 1459 raft_consensus.cc:2229] T 3e6246af76a443c0ab45f5d0ee94057c P 0cd27c0909644c38991b15cd2c5bc574 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 15:11:32.456287 1459 raft_consensus.cc:2258] T 3e6246af76a443c0ab45f5d0ee94057c P 0cd27c0909644c38991b15cd2c5bc574 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:11:32.456439 1462 ts_tablet_manager.cc:1905] T 3e6246af76a443c0ab45f5d0ee94057c P 5a1ab2d4360447afa9f51e9c649e3eb1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:11:32.456445 1460 ts_tablet_manager.cc:1905] T 3e6246af76a443c0ab45f5d0ee94057c P cf43c478316e4b46a30860f95d7997b2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:11:32.456925 1459 ts_tablet_manager.cc:1905] T 3e6246af76a443c0ab45f5d0ee94057c P 0cd27c0909644c38991b15cd2c5bc574: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:11:32.457589 781 tablet_service.cc:1467] Processing CreateTablet for tablet e3c1bc4996d542e88d3fa5cc0622a2b9 (DEFAULT_TABLE table=test-table [id=8b5b6bf85d2f4548ac87a2a09afe90a2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:11:32.457710 781 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e3c1bc4996d542e88d3fa5cc0622a2b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:11:32.458491 1460 ts_tablet_manager.cc:1918] T 3e6246af76a443c0ab45f5d0ee94057c P cf43c478316e4b46a30860f95d7997b2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 15:11:32.458545 1460 log.cc:1196] T 3e6246af76a443c0ab45f5d0ee94057c P cf43c478316e4b46a30860f95d7997b2: Deleting WAL directory at /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115480851507-30472-0/minicluster-data/ts-2/wal/wals/3e6246af76a443c0ab45f5d0ee94057c
I20260807 15:11:32.458789 1460 ts_tablet_manager.cc:1939] T 3e6246af76a443c0ab45f5d0ee94057c P cf43c478316e4b46a30860f95d7997b2: Deleting consensus metadata
I20260807 15:11:32.458861 1459 ts_tablet_manager.cc:1918] T 3e6246af76a443c0ab45f5d0ee94057c P 0cd27c0909644c38991b15cd2c5bc574: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 15:11:32.458912 1459 log.cc:1196] T 3e6246af76a443c0ab45f5d0ee94057c P 0cd27c0909644c38991b15cd2c5bc574: Deleting WAL directory at /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115480851507-30472-0/minicluster-data/ts-0/wal/wals/3e6246af76a443c0ab45f5d0ee94057c
I20260807 15:11:32.459003 1475 tablet_bootstrap.cc:492] T e3c1bc4996d542e88d3fa5cc0622a2b9 P 0cd27c0909644c38991b15cd2c5bc574: Bootstrap starting.
I20260807 15:11:32.458999 1462 ts_tablet_manager.cc:1918] T 3e6246af76a443c0ab45f5d0ee94057c P 5a1ab2d4360447afa9f51e9c649e3eb1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 15:11:32.459052 1462 log.cc:1196] T 3e6246af76a443c0ab45f5d0ee94057c P 5a1ab2d4360447afa9f51e9c649e3eb1: Deleting WAL directory at /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115480851507-30472-0/minicluster-data/ts-1/wal/wals/3e6246af76a443c0ab45f5d0ee94057c
I20260807 15:11:32.459177 1459 ts_tablet_manager.cc:1939] T 3e6246af76a443c0ab45f5d0ee94057c P 0cd27c0909644c38991b15cd2c5bc574: Deleting consensus metadata
I20260807 15:11:32.459270 1462 ts_tablet_manager.cc:1939] T 3e6246af76a443c0ab45f5d0ee94057c P 5a1ab2d4360447afa9f51e9c649e3eb1: Deleting consensus metadata
I20260807 15:11:32.459437 1475 tablet_bootstrap.cc:654] T e3c1bc4996d542e88d3fa5cc0622a2b9 P 0cd27c0909644c38991b15cd2c5bc574: Neither blocks nor log segments found. Creating new log.
I20260807 15:11:32.459903 555 catalog_manager.cc:4841] TS 0cd27c0909644c38991b15cd2c5bc574 (127.29.194.1:34761): tablet 3e6246af76a443c0ab45f5d0ee94057c (table test-table [id=2267f5ef59204ac7808f0ee39c7da91e]) successfully deleted
I20260807 15:11:32.460078 1475 tablet_bootstrap.cc:492] T e3c1bc4996d542e88d3fa5cc0622a2b9 P 0cd27c0909644c38991b15cd2c5bc574: No bootstrap required, opened a new log
I20260807 15:11:32.460124 1475 ts_tablet_manager.cc:1397] T e3c1bc4996d542e88d3fa5cc0622a2b9 P 0cd27c0909644c38991b15cd2c5bc574: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260807 15:11:32.460237 1475 raft_consensus.cc:348] T e3c1bc4996d542e88d3fa5cc0622a2b9 P 0cd27c0909644c38991b15cd2c5bc574 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0cd27c0909644c38991b15cd2c5bc574" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 34761 } } peers { permanent_uuid: "5a1ab2d4360447afa9f51e9c649e3eb1" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 37537 } } peers { permanent_uuid: "cf43c478316e4b46a30860f95d7997b2" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 33731 } }
I20260807 15:11:32.460301 1475 raft_consensus.cc:374] T e3c1bc4996d542e88d3fa5cc0622a2b9 P 0cd27c0909644c38991b15cd2c5bc574 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:11:32.460318 1475 raft_consensus.cc:729] T e3c1bc4996d542e88d3fa5cc0622a2b9 P 0cd27c0909644c38991b15cd2c5bc574 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0cd27c0909644c38991b15cd2c5bc574, State: Initialized, Role: FOLLOWER
I20260807 15:11:32.460367 1475 consensus_queue.cc:260] T e3c1bc4996d542e88d3fa5cc0622a2b9 P 0cd27c0909644c38991b15cd2c5bc574 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0cd27c0909644c38991b15cd2c5bc574" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 34761 } } peers { permanent_uuid: "5a1ab2d4360447afa9f51e9c649e3eb1" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 37537 } } peers { permanent_uuid: "cf43c478316e4b46a30860f95d7997b2" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 33731 } }
I20260807 15:11:32.460467 1475 ts_tablet_manager.cc:1428] T e3c1bc4996d542e88d3fa5cc0622a2b9 P 0cd27c0909644c38991b15cd2c5bc574: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260807 15:11:32.460666 556 catalog_manager.cc:4841] TS cf43c478316e4b46a30860f95d7997b2 (127.29.194.3:33731): tablet 3e6246af76a443c0ab45f5d0ee94057c (table test-table [id=2267f5ef59204ac7808f0ee39c7da91e]) successfully deleted
I20260807 15:11:32.460723 556 catalog_manager.cc:4841] TS 5a1ab2d4360447afa9f51e9c649e3eb1 (127.29.194.2:37537): tablet 3e6246af76a443c0ab45f5d0ee94057c (table test-table [id=2267f5ef59204ac7808f0ee39c7da91e]) successfully deleted
I20260807 15:11:32.460788 998 tablet_service.cc:1467] Processing CreateTablet for tablet e3c1bc4996d542e88d3fa5cc0622a2b9 (DEFAULT_TABLE table=test-table [id=8b5b6bf85d2f4548ac87a2a09afe90a2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:11:32.460779 1205 tablet_service.cc:1467] Processing CreateTablet for tablet e3c1bc4996d542e88d3fa5cc0622a2b9 (DEFAULT_TABLE table=test-table [id=8b5b6bf85d2f4548ac87a2a09afe90a2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:11:32.460898 998 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e3c1bc4996d542e88d3fa5cc0622a2b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:11:32.460904 1205 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e3c1bc4996d542e88d3fa5cc0622a2b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:11:32.461983 1474 tablet_bootstrap.cc:492] T e3c1bc4996d542e88d3fa5cc0622a2b9 P 5a1ab2d4360447afa9f51e9c649e3eb1: Bootstrap starting.
I20260807 15:11:32.462380 1474 tablet_bootstrap.cc:654] T e3c1bc4996d542e88d3fa5cc0622a2b9 P 5a1ab2d4360447afa9f51e9c649e3eb1: Neither blocks nor log segments found. Creating new log.
I20260807 15:11:32.462724 1473 tablet_bootstrap.cc:492] T e3c1bc4996d542e88d3fa5cc0622a2b9 P cf43c478316e4b46a30860f95d7997b2: Bootstrap starting.
I20260807 15:11:32.463089 1473 tablet_bootstrap.cc:654] T e3c1bc4996d542e88d3fa5cc0622a2b9 P cf43c478316e4b46a30860f95d7997b2: Neither blocks nor log segments found. Creating new log.
I20260807 15:11:32.463109 1474 tablet_bootstrap.cc:492] T e3c1bc4996d542e88d3fa5cc0622a2b9 P 5a1ab2d4360447afa9f51e9c649e3eb1: No bootstrap required, opened a new log
I20260807 15:11:32.463184 1474 ts_tablet_manager.cc:1397] T e3c1bc4996d542e88d3fa5cc0622a2b9 P 5a1ab2d4360447afa9f51e9c649e3eb1: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260807 15:11:32.463315 1474 raft_consensus.cc:348] T e3c1bc4996d542e88d3fa5cc0622a2b9 P 5a1ab2d4360447afa9f51e9c649e3eb1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0cd27c0909644c38991b15cd2c5bc574" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 34761 } } peers { permanent_uuid: "5a1ab2d4360447afa9f51e9c649e3eb1" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 37537 } } peers { permanent_uuid: "cf43c478316e4b46a30860f95d7997b2" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 33731 } }
I20260807 15:11:32.463397 1474 raft_consensus.cc:374] T e3c1bc4996d542e88d3fa5cc0622a2b9 P 5a1ab2d4360447afa9f51e9c649e3eb1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:11:32.463411 1474 raft_consensus.cc:729] T e3c1bc4996d542e88d3fa5cc0622a2b9 P 5a1ab2d4360447afa9f51e9c649e3eb1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a1ab2d4360447afa9f51e9c649e3eb1, State: Initialized, Role: FOLLOWER
I20260807 15:11:32.463469 1474 consensus_queue.cc:260] T e3c1bc4996d542e88d3fa5cc0622a2b9 P 5a1ab2d4360447afa9f51e9c649e3eb1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0cd27c0909644c38991b15cd2c5bc574" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 34761 } } peers { permanent_uuid: "5a1ab2d4360447afa9f51e9c649e3eb1" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 37537 } } peers { permanent_uuid: "cf43c478316e4b46a30860f95d7997b2" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 33731 } }
I20260807 15:11:32.463579 1474 ts_tablet_manager.cc:1428] T e3c1bc4996d542e88d3fa5cc0622a2b9 P 5a1ab2d4360447afa9f51e9c649e3eb1: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260807 15:11:32.463685 1473 tablet_bootstrap.cc:492] T e3c1bc4996d542e88d3fa5cc0622a2b9 P cf43c478316e4b46a30860f95d7997b2: No bootstrap required, opened a new log
I20260807 15:11:32.463747 1473 ts_tablet_manager.cc:1397] T e3c1bc4996d542e88d3fa5cc0622a2b9 P cf43c478316e4b46a30860f95d7997b2: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260807 15:11:32.463888 1473 raft_consensus.cc:348] T e3c1bc4996d542e88d3fa5cc0622a2b9 P cf43c478316e4b46a30860f95d7997b2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0cd27c0909644c38991b15cd2c5bc574" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 34761 } } peers { permanent_uuid: "5a1ab2d4360447afa9f51e9c649e3eb1" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 37537 } } peers { permanent_uuid: "cf43c478316e4b46a30860f95d7997b2" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 33731 } }
I20260807 15:11:32.463971 1473 raft_consensus.cc:374] T e3c1bc4996d542e88d3fa5cc0622a2b9 P cf43c478316e4b46a30860f95d7997b2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:11:32.463990 1473 raft_consensus.cc:729] T e3c1bc4996d542e88d3fa5cc0622a2b9 P cf43c478316e4b46a30860f95d7997b2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf43c478316e4b46a30860f95d7997b2, State: Initialized, Role: FOLLOWER
I20260807 15:11:32.464052 1473 consensus_queue.cc:260] T e3c1bc4996d542e88d3fa5cc0622a2b9 P cf43c478316e4b46a30860f95d7997b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0cd27c0909644c38991b15cd2c5bc574" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 34761 } } peers { permanent_uuid: "5a1ab2d4360447afa9f51e9c649e3eb1" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 37537 } } peers { permanent_uuid: "cf43c478316e4b46a30860f95d7997b2" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 33731 } }
I20260807 15:11:32.464164 1473 ts_tablet_manager.cc:1428] T e3c1bc4996d542e88d3fa5cc0622a2b9 P cf43c478316e4b46a30860f95d7997b2: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260807 15:11:32.625507 1483 raft_consensus.cc:482] T e3c1bc4996d542e88d3fa5cc0622a2b9 P 5a1ab2d4360447afa9f51e9c649e3eb1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 15:11:32.625643 1483 raft_consensus.cc:504] T e3c1bc4996d542e88d3fa5cc0622a2b9 P 5a1ab2d4360447afa9f51e9c649e3eb1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0cd27c0909644c38991b15cd2c5bc574" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 34761 } } peers { permanent_uuid: "5a1ab2d4360447afa9f51e9c649e3eb1" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 37537 } } peers { permanent_uuid: "cf43c478316e4b46a30860f95d7997b2" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 33731 } }
I20260807 15:11:32.625837 1483 leader_election.cc:288] T e3c1bc4996d542e88d3fa5cc0622a2b9 P 5a1ab2d4360447afa9f51e9c649e3eb1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0cd27c0909644c38991b15cd2c5bc574 (127.29.194.1:34761), cf43c478316e4b46a30860f95d7997b2 (127.29.194.3:33731)
I20260807 15:11:32.626137 817 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e3c1bc4996d542e88d3fa5cc0622a2b9" candidate_uuid: "5a1ab2d4360447afa9f51e9c649e3eb1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0cd27c0909644c38991b15cd2c5bc574" is_pre_election: true
I20260807 15:11:32.626236 817 raft_consensus.cc:2454] T e3c1bc4996d542e88d3fa5cc0622a2b9 P 0cd27c0909644c38991b15cd2c5bc574 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a1ab2d4360447afa9f51e9c649e3eb1 in term 0.
I20260807 15:11:32.626320 1229 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e3c1bc4996d542e88d3fa5cc0622a2b9" candidate_uuid: "5a1ab2d4360447afa9f51e9c649e3eb1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf43c478316e4b46a30860f95d7997b2" is_pre_election: true
I20260807 15:11:32.626430 923 leader_election.cc:302] T e3c1bc4996d542e88d3fa5cc0622a2b9 P 5a1ab2d4360447afa9f51e9c649e3eb1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0cd27c0909644c38991b15cd2c5bc574, 5a1ab2d4360447afa9f51e9c649e3eb1; no voters:
I20260807 15:11:32.626523 1229 raft_consensus.cc:2454] T e3c1bc4996d542e88d3fa5cc0622a2b9 P cf43c478316e4b46a30860f95d7997b2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a1ab2d4360447afa9f51e9c649e3eb1 in term 0.
I20260807 15:11:32.626602 1483 raft_consensus.cc:2781] T e3c1bc4996d542e88d3fa5cc0622a2b9 P 5a1ab2d4360447afa9f51e9c649e3eb1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 15:11:32.626672 1483 raft_consensus.cc:482] T e3c1bc4996d542e88d3fa5cc0622a2b9 P 5a1ab2d4360447afa9f51e9c649e3eb1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 15:11:32.626693 1483 raft_consensus.cc:3037] T e3c1bc4996d542e88d3fa5cc0622a2b9 P 5a1ab2d4360447afa9f51e9c649e3eb1 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:11:32.627547 1483 raft_consensus.cc:504] T e3c1bc4996d542e88d3fa5cc0622a2b9 P 5a1ab2d4360447afa9f51e9c649e3eb1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0cd27c0909644c38991b15cd2c5bc574" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 34761 } } peers { permanent_uuid: "5a1ab2d4360447afa9f51e9c649e3eb1" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 37537 } } peers { permanent_uuid: "cf43c478316e4b46a30860f95d7997b2" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 33731 } }
I20260807 15:11:32.627701 1483 leader_election.cc:288] T e3c1bc4996d542e88d3fa5cc0622a2b9 P 5a1ab2d4360447afa9f51e9c649e3eb1 [CANDIDATE]: Term 1 election: Requested vote from peers 0cd27c0909644c38991b15cd2c5bc574 (127.29.194.1:34761), cf43c478316e4b46a30860f95d7997b2 (127.29.194.3:33731)
I20260807 15:11:32.627921 817 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e3c1bc4996d542e88d3fa5cc0622a2b9" candidate_uuid: "5a1ab2d4360447afa9f51e9c649e3eb1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0cd27c0909644c38991b15cd2c5bc574"
I20260807 15:11:32.628026 817 raft_consensus.cc:3037] T e3c1bc4996d542e88d3fa5cc0622a2b9 P 0cd27c0909644c38991b15cd2c5bc574 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:11:32.627980 1229 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e3c1bc4996d542e88d3fa5cc0622a2b9" candidate_uuid: "5a1ab2d4360447afa9f51e9c649e3eb1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf43c478316e4b46a30860f95d7997b2"
I20260807 15:11:32.628067 1229 raft_consensus.cc:3037] T e3c1bc4996d542e88d3fa5cc0622a2b9 P cf43c478316e4b46a30860f95d7997b2 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:11:32.628718 1229 raft_consensus.cc:2454] T e3c1bc4996d542e88d3fa5cc0622a2b9 P cf43c478316e4b46a30860f95d7997b2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a1ab2d4360447afa9f51e9c649e3eb1 in term 1.
I20260807 15:11:32.628767 817 raft_consensus.cc:2454] T e3c1bc4996d542e88d3fa5cc0622a2b9 P 0cd27c0909644c38991b15cd2c5bc574 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a1ab2d4360447afa9f51e9c649e3eb1 in term 1.
I20260807 15:11:32.628964 923 leader_election.cc:302] T e3c1bc4996d542e88d3fa5cc0622a2b9 P 5a1ab2d4360447afa9f51e9c649e3eb1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0cd27c0909644c38991b15cd2c5bc574, 5a1ab2d4360447afa9f51e9c649e3eb1; no voters:
I20260807 15:11:32.629065 1483 raft_consensus.cc:2781] T e3c1bc4996d542e88d3fa5cc0622a2b9 P 5a1ab2d4360447afa9f51e9c649e3eb1 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:11:32.629129 1483 raft_consensus.cc:686] T e3c1bc4996d542e88d3fa5cc0622a2b9 P 5a1ab2d4360447afa9f51e9c649e3eb1 [term 1 LEADER]: Becoming Leader. State: Replica: 5a1ab2d4360447afa9f51e9c649e3eb1, State: Running, Role: LEADER
I20260807 15:11:32.629186 1483 consensus_queue.cc:237] T e3c1bc4996d542e88d3fa5cc0622a2b9 P 5a1ab2d4360447afa9f51e9c649e3eb1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0cd27c0909644c38991b15cd2c5bc574" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 34761 } } peers { permanent_uuid: "5a1ab2d4360447afa9f51e9c649e3eb1" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 37537 } } peers { permanent_uuid: "cf43c478316e4b46a30860f95d7997b2" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 33731 } }
I20260807 15:11:32.629796 571 catalog_manager.cc:5495] T e3c1bc4996d542e88d3fa5cc0622a2b9 P 5a1ab2d4360447afa9f51e9c649e3eb1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5a1ab2d4360447afa9f51e9c649e3eb1 (127.29.194.2). New cstate: current_term: 1 leader_uuid: "5a1ab2d4360447afa9f51e9c649e3eb1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0cd27c0909644c38991b15cd2c5bc574" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 34761 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5a1ab2d4360447afa9f51e9c649e3eb1" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 37537 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cf43c478316e4b46a30860f95d7997b2" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 33731 } health_report { overall_health: UNKNOWN } } }
I20260807 15:11:32.654103 817 raft_consensus.cc:1261] T e3c1bc4996d542e88d3fa5cc0622a2b9 P 0cd27c0909644c38991b15cd2c5bc574 [term 1 FOLLOWER]: Refusing update from remote peer 5a1ab2d4360447afa9f51e9c649e3eb1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:11:32.654363 1229 raft_consensus.cc:1261] T e3c1bc4996d542e88d3fa5cc0622a2b9 P cf43c478316e4b46a30860f95d7997b2 [term 1 FOLLOWER]: Refusing update from remote peer 5a1ab2d4360447afa9f51e9c649e3eb1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:11:32.654341 1483 consensus_queue.cc:1035] T e3c1bc4996d542e88d3fa5cc0622a2b9 P 5a1ab2d4360447afa9f51e9c649e3eb1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0cd27c0909644c38991b15cd2c5bc574" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 34761 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:11:32.654562 1483 consensus_queue.cc:1035] T e3c1bc4996d542e88d3fa5cc0622a2b9 P 5a1ab2d4360447afa9f51e9c649e3eb1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf43c478316e4b46a30860f95d7997b2" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 33731 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260807 15:11:32.658838 1436 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260807 15:11:34.180580 572 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40200:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 15:11:34.180696 572 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40200:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 15:11:34.181599 572 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 86993e10419047948fdd2437d6f0c4fc: Sending DeleteTablet for 3 replicas of tablet e3c1bc4996d542e88d3fa5cc0622a2b9
I20260807 15:11:34.181840 998 tablet_service.cc:1514] Processing DeleteTablet for tablet e3c1bc4996d542e88d3fa5cc0622a2b9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:11:34 UTC) from {username='kudu', principal='kudu/127.29.194.62@KRBTEST.COM'} at 127.0.0.1:52208
I20260807 15:11:34.181862 781 tablet_service.cc:1514] Processing DeleteTablet for tablet e3c1bc4996d542e88d3fa5cc0622a2b9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:11:34 UTC) from {username='kudu', principal='kudu/127.29.194.62@KRBTEST.COM'} at 127.0.0.1:53104
I20260807 15:11:34.181861 1205 tablet_service.cc:1514] Processing DeleteTablet for tablet e3c1bc4996d542e88d3fa5cc0622a2b9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:11:34 UTC) from {username='kudu', principal='kudu/127.29.194.62@KRBTEST.COM'} at 127.0.0.1:55238
I20260807 15:11:34.182052 30472 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7fZSIL/build/release/bin/kudu with pid 687
I20260807 15:11:34.189219 1540 tablet_replica.cc:330] T e3c1bc4996d542e88d3fa5cc0622a2b9 P 5a1ab2d4360447afa9f51e9c649e3eb1: stopping tablet replica
I20260807 15:11:34.189337 1540 raft_consensus.cc:2229] T e3c1bc4996d542e88d3fa5cc0622a2b9 P 5a1ab2d4360447afa9f51e9c649e3eb1 [term 1 LEADER]: Raft consensus shutting down.
I20260807 15:11:34.189414 1540 raft_consensus.cc:2258] T e3c1bc4996d542e88d3fa5cc0622a2b9 P 5a1ab2d4360447afa9f51e9c649e3eb1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:11:34.189719 1540 ts_tablet_manager.cc:1905] T e3c1bc4996d542e88d3fa5cc0622a2b9 P 5a1ab2d4360447afa9f51e9c649e3eb1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:11:34.191494 1540 ts_tablet_manager.cc:1918] T e3c1bc4996d542e88d3fa5cc0622a2b9 P 5a1ab2d4360447afa9f51e9c649e3eb1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260807 15:11:34.191569 1540 log.cc:1196] T e3c1bc4996d542e88d3fa5cc0622a2b9 P 5a1ab2d4360447afa9f51e9c649e3eb1: Deleting WAL directory at /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115480851507-30472-0/minicluster-data/ts-1/wal/wals/e3c1bc4996d542e88d3fa5cc0622a2b9
I20260807 15:11:34.191743 1544 tablet_replica.cc:330] T e3c1bc4996d542e88d3fa5cc0622a2b9 P cf43c478316e4b46a30860f95d7997b2: stopping tablet replica
I20260807 15:11:34.191819 1540 ts_tablet_manager.cc:1939] T e3c1bc4996d542e88d3fa5cc0622a2b9 P 5a1ab2d4360447afa9f51e9c649e3eb1: Deleting consensus metadata
I20260807 15:11:34.191836 1544 raft_consensus.cc:2229] T e3c1bc4996d542e88d3fa5cc0622a2b9 P cf43c478316e4b46a30860f95d7997b2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 15:11:34.191888 1544 raft_consensus.cc:2258] T e3c1bc4996d542e88d3fa5cc0622a2b9 P cf43c478316e4b46a30860f95d7997b2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:11:34.192180 1544 ts_tablet_manager.cc:1905] T e3c1bc4996d542e88d3fa5cc0622a2b9 P cf43c478316e4b46a30860f95d7997b2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:11:34.192356 556 catalog_manager.cc:4841] TS 5a1ab2d4360447afa9f51e9c649e3eb1 (127.29.194.2:37537): tablet e3c1bc4996d542e88d3fa5cc0622a2b9 (table test-table [id=8b5b6bf85d2f4548ac87a2a09afe90a2]) successfully deleted
I20260807 15:11:34.193569 1544 ts_tablet_manager.cc:1918] T e3c1bc4996d542e88d3fa5cc0622a2b9 P cf43c478316e4b46a30860f95d7997b2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260807 15:11:34.193624 1544 log.cc:1196] T e3c1bc4996d542e88d3fa5cc0622a2b9 P cf43c478316e4b46a30860f95d7997b2: Deleting WAL directory at /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115480851507-30472-0/minicluster-data/ts-2/wal/wals/e3c1bc4996d542e88d3fa5cc0622a2b9
I20260807 15:11:34.193840 1544 ts_tablet_manager.cc:1939] T e3c1bc4996d542e88d3fa5cc0622a2b9 P cf43c478316e4b46a30860f95d7997b2: Deleting consensus metadata
I20260807 15:11:34.194311 556 catalog_manager.cc:4841] TS cf43c478316e4b46a30860f95d7997b2 (127.29.194.3:33731): tablet e3c1bc4996d542e88d3fa5cc0622a2b9 (table test-table [id=8b5b6bf85d2f4548ac87a2a09afe90a2]) successfully deleted
W20260807 15:11:34.196209 555 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.29.194.1:34761 (error 108)
I20260807 15:11:34.196511 30472 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7fZSIL/build/release/bin/kudu with pid 895
I20260807 15:11:34.207161 1546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:34.196610 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.1:34761 (local address 127.0.0.1:53126)
0807 15:11:34.207012 (+ 10402us) negotiation.cc:107] Waiting for socket to connect
0807 15:11:34.207099 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.194.1:34761: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":10353,"thread_start_us":61,"threads_started":1}
W20260807 15:11:34.207381 555 catalog_manager.cc:4584] TS 0cd27c0909644c38991b15cd2c5bc574 (127.29.194.1:34761): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet e3c1bc4996d542e88d3fa5cc0622a2b9: Network error: Client connection negotiation failed: client connection to 127.29.194.1:34761: connect: Connection refused (error 111)
I20260807 15:11:34.245743 30472 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7fZSIL/build/release/bin/kudu with pid 1106
I20260807 15:11:34.260211 1546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:34.259874 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.1:34761 (local address 127.0.0.1:53134)
0807 15:11:34.260057 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0807 15:11:34.260135 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.194.1:34761: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114}
I20260807 15:11:34.260432 30472 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7fZSIL/build/release/bin/kudu with pid 525
2026-08-07T15:11:34Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (5975 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115480851507-30472-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 15:11:34 dist-test-slave-42z9 krb5kdc[1552](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 15:11:34 dist-test-slave-42z9 krb5kdc[1552](info): set up 2 sockets
Aug 07 15:11:34 dist-test-slave-42z9 krb5kdc[1552](info): commencing operation
krb5kdc: starting...
W20260807 15:11:35.429711 30472 mini_kdc.cc:121] Time spent starting KDC: real 1.139s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115480851507-30472-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115480851507-30472-0/krb5kdc/test-user.keytab.
Aug 07 15:11:35 dist-test-slave-42z9 krb5kdc[1552](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115495, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T15:11:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T15:11:35Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.194.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.194.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.194.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.62.keytab.
Entry for principal kudu/127.29.194.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.62.keytab.
I20260807 15:11:35.568975 30472 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7fZSIL/build/release/bin/kudu
/tmp/dist-test-task7fZSIL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115480851507-30472-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115480851507-30472-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115480851507-30472-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115480851507-30472-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.194.62:39733
--webserver_interface=127.29.194.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.62.keytab
--principal=kudu/127.29.194.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.194.20:37959
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.194.62:39733
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115480851507-30472-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115480851507-30472-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115480851507-30472-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:11:35.654512 1566 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:11:35.654767 1566 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:11:35.654801 1566 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:11:35.656059 1566 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:11:35.656114 1566 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:11:35.656131 1566 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:11:35.656148 1566 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:11:35.656163 1566 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:11:35.657864 1566 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.194.20:37959
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115480851507-30472-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115480851507-30472-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.194.62:39733
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.62.keytab
--principal=kudu/127.29.194.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.194.62:39733
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115480851507-30472-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.194.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1566
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115480851507-30472-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:11:35.658159 1566 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:11:35.658392 1566 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:11:35.663836 1577 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:11:35.663851 1575 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:11:35.663861 1566 server_base.cc:1034] running on GCE node
W20260807 15:11:35.663836 1574 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:11:35.664317 1566 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:11:35.664669 1566 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:11:35.665863 1566 hybrid_clock.cc:648] HybridClock initialized: now 1786115495665798 us; error 92 us; skew 500 ppm
Aug 07 15:11:35 dist-test-slave-42z9 krb5kdc[1552](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115495, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.194.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:11:35.668502 1566 init.cc:377] Logged in from keytab as kudu/127.29.194.62@KRBTEST.COM (short username kudu)
I20260807 15:11:35.668965 1566 webserver.cc:460] Webserver started at http://127.29.194.62:45665/ using document root <none> and password file <none>
I20260807 15:11:35.669200 1566 fs_manager.cc:359] Metadata directory not provided
I20260807 15:11:35.669250 1566 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:11:35.669378 1566 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:11:35.670466 1566 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115480851507-30472-0/minicluster-data/master-0/data/instance:
uuid: "863c013959b74ecba8c1571aedd64454"
format_stamp: "Formatted at 2026-08-07 15:11:35 on dist-test-slave-42z9"
server_key: "bac2cbf27180881ff3544775522ff0fd"
server_key_iv: "c6f1cba11313ef963caf498feac013b3"
server_key_version: "encryptionkey@0"
I20260807 15:11:35.670789 1566 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115480851507-30472-0/minicluster-data/master-0/wal/instance:
uuid: "863c013959b74ecba8c1571aedd64454"
format_stamp: "Formatted at 2026-08-07 15:11:35 on dist-test-slave-42z9"
server_key: "bac2cbf27180881ff3544775522ff0fd"
server_key_iv: "c6f1cba11313ef963caf498feac013b3"
server_key_version: "encryptionkey@0"
I20260807 15:11:35.672132 1566 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 15:11:35.673003 1585 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:11:35.673198 1566 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:11:35.673290 1566 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115480851507-30472-0/minicluster-data/master-0/data,/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115480851507-30472-0/minicluster-data/master-0/wal
uuid: "863c013959b74ecba8c1571aedd64454"
format_stamp: "Formatted at 2026-08-07 15:11:35 on dist-test-slave-42z9"
server_key: "bac2cbf27180881ff3544775522ff0fd"
server_key_iv: "c6f1cba11313ef963caf498feac013b3"
server_key_version: "encryptionkey@0"
I20260807 15:11:35.673357 1566 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115480851507-30472-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115480851507-30472-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115480851507-30472-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:11:35.702658 1566 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:11:35.705022 1566 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:11:35.705194 1566 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:11:35.709270 1566 rpc_server.cc:307] RPC server started. Bound to: 127.29.194.62:39733
I20260807 15:11:35.709321 1664 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.194.62:39733 every 8 connection(s)
I20260807 15:11:35.709709 1566 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115480851507-30472-0/minicluster-data/master-0/data/info.pb
I20260807 15:11:35.710702 1666 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:11:35.713179 1666 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 863c013959b74ecba8c1571aedd64454: Bootstrap starting.
I20260807 15:11:35.713877 1666 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 863c013959b74ecba8c1571aedd64454: Neither blocks nor log segments found. Creating new log.
I20260807 15:11:35.714207 1666 log.cc:824] T 00000000000000000000000000000000 P 863c013959b74ecba8c1571aedd64454: Log is configured to *not* fsync() on all Append() calls
I20260807 15:11:35.715312 1666 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 863c013959b74ecba8c1571aedd64454: No bootstrap required, opened a new log
I20260807 15:11:35.716002 30472 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7fZSIL/build/release/bin/kudu as pid 1566
I20260807 15:11:35.716120 30472 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115480851507-30472-0/minicluster-data/master-0/wal/instance
I20260807 15:11:35.716293 30472 external_mini_cluster.cc:1428] Setting key 90e8e1d85baaa235d97e6d5f7805dad7
I20260807 15:11:35.718045 1666 raft_consensus.cc:348] T 00000000000000000000000000000000 P 863c013959b74ecba8c1571aedd64454 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "863c013959b74ecba8c1571aedd64454" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 39733 } }
I20260807 15:11:35.718310 1666 raft_consensus.cc:374] T 00000000000000000000000000000000 P 863c013959b74ecba8c1571aedd64454 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:11:35.718353 1666 raft_consensus.cc:729] T 00000000000000000000000000000000 P 863c013959b74ecba8c1571aedd64454 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 863c013959b74ecba8c1571aedd64454, State: Initialized, Role: FOLLOWER
I20260807 15:11:35.718561 1666 consensus_queue.cc:260] T 00000000000000000000000000000000 P 863c013959b74ecba8c1571aedd64454 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "863c013959b74ecba8c1571aedd64454" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 39733 } }
I20260807 15:11:35.718667 1666 raft_consensus.cc:388] T 00000000000000000000000000000000 P 863c013959b74ecba8c1571aedd64454 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:11:35.718693 1666 raft_consensus.cc:482] T 00000000000000000000000000000000 P 863c013959b74ecba8c1571aedd64454 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:11:35.718724 1666 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 863c013959b74ecba8c1571aedd64454 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:11:35.720286 1666 raft_consensus.cc:504] T 00000000000000000000000000000000 P 863c013959b74ecba8c1571aedd64454 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "863c013959b74ecba8c1571aedd64454" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 39733 } }
I20260807 15:11:35.720420 1666 leader_election.cc:302] T 00000000000000000000000000000000 P 863c013959b74ecba8c1571aedd64454 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 863c013959b74ecba8c1571aedd64454; no voters:
I20260807 15:11:35.720670 1666 leader_election.cc:288] T 00000000000000000000000000000000 P 863c013959b74ecba8c1571aedd64454 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 15:11:35.720804 1672 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 863c013959b74ecba8c1571aedd64454 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:11:35.720996 1666 sys_catalog.cc:564] T 00000000000000000000000000000000 P 863c013959b74ecba8c1571aedd64454 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:11:35.721097 1672 raft_consensus.cc:686] T 00000000000000000000000000000000 P 863c013959b74ecba8c1571aedd64454 [term 1 LEADER]: Becoming Leader. State: Replica: 863c013959b74ecba8c1571aedd64454, State: Running, Role: LEADER
I20260807 15:11:35.721287 1672 consensus_queue.cc:237] T 00000000000000000000000000000000 P 863c013959b74ecba8c1571aedd64454 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "863c013959b74ecba8c1571aedd64454" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 39733 } }
I20260807 15:11:35.721839 1673 sys_catalog.cc:455] T 00000000000000000000000000000000 P 863c013959b74ecba8c1571aedd64454 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "863c013959b74ecba8c1571aedd64454" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "863c013959b74ecba8c1571aedd64454" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 39733 } } }
I20260807 15:11:35.721941 1673 sys_catalog.cc:458] T 00000000000000000000000000000000 P 863c013959b74ecba8c1571aedd64454 [sys.catalog]: This master's current role is: LEADER
I20260807 15:11:35.722333 1674 sys_catalog.cc:455] T 00000000000000000000000000000000 P 863c013959b74ecba8c1571aedd64454 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 863c013959b74ecba8c1571aedd64454. Latest consensus state: current_term: 1 leader_uuid: "863c013959b74ecba8c1571aedd64454" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "863c013959b74ecba8c1571aedd64454" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 39733 } } }
I20260807 15:11:35.722404 1674 sys_catalog.cc:458] T 00000000000000000000000000000000 P 863c013959b74ecba8c1571aedd64454 [sys.catalog]: This master's current role is: LEADER
I20260807 15:11:35.722890 1685 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:11:35.723702 1685 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 07 15:11:35 dist-test-slave-42z9 krb5kdc[1552](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115495, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.194.62@KRBTEST.COM
I20260807 15:11:35.725968 1685 catalog_manager.cc:1347] Generated new cluster ID: 35f1ed2635fb4f04af3480542680f20a
I20260807 15:11:35.726034 1685 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:11:35.730981 1670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:35.716996 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38446 (local address 127.29.194.62:39733)
0807 15:11:35.717275 (+ 279us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:35.717289 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:35.717335 (+ 46us) server_negotiation.cc:407] Connection header received
0807 15:11:35.717974 (+ 639us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:35.717987 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:35.718073 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:35.718303 (+ 230us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:11:35.719303 (+ 1000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:35.720844 (+ 1541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:35.722050 (+ 1206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:35.722768 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:35.723275 (+ 507us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:35.725924 (+ 2649us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:11:35.725929 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:11:35.725929 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:11:35.725942 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:11:35.728392 (+ 2450us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:35.728782 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:35.728786 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:35.728786 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:35.728934 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:35.729118 (+ 184us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:35.729119 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:35.729119 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:35.729430 (+ 311us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:11:35.729612 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0807 15:11:35.730463 (+ 851us) server_negotiation.cc:299] Negotiation successful
0807 15:11:35.730690 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":209,"thread_start_us":124,"threads_started":1}
I20260807 15:11:35.749257 1685 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:11:35.750133 1685 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:11:35.759308 1685 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 863c013959b74ecba8c1571aedd64454: Generated new TSK 0
I20260807 15:11:35.759622 1685 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.29.194.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.194.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.194.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.1.keytab.
Entry for principal kudu/127.29.194.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.1.keytab.
I20260807 15:11:35.797982 30472 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7fZSIL/build/release/bin/kudu
/tmp/dist-test-task7fZSIL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115480851507-30472-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115480851507-30472-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115480851507-30472-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115480851507-30472-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.194.1:0
--local_ip_for_outbound_sockets=127.29.194.1
--webserver_interface=127.29.194.1
--webserver_port=0
--tserver_master_addrs=127.29.194.62:39733
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.1.keytab
--principal=kudu/127.29.194.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.194.20:37959
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115480851507-30472-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115480851507-30472-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115480851507-30472-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:11:35.888845 1702 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:11:35.889076 1702 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:11:35.889102 1702 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:11:35.890419 1702 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:11:35.890476 1702 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:11:35.890506 1702 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.194.1
I20260807 15:11:35.892295 1702 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.194.20:37959
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115480851507-30472-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115480851507-30472-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.1.keytab
--principal=kudu/127.29.194.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.194.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115480851507-30472-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.194.1
--webserver_port=0
--tserver_master_addrs=127.29.194.62:39733
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1702
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.194.1
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115480851507-30472-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:11:35.892592 1702 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:11:35.892851 1702 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:11:35.898574 1713 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:11:35.898629 1711 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:11:35.898766 1702 server_base.cc:1034] running on GCE node
W20260807 15:11:35.898662 1708 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:11:35.899505 1702 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:11:35.899758 1702 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:11:35.900938 1702 hybrid_clock.cc:648] HybridClock initialized: now 1786115495900871 us; error 104 us; skew 500 ppm
Aug 07 15:11:35 dist-test-slave-42z9 krb5kdc[1552](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115495, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.194.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:11:35.904126 1702 init.cc:377] Logged in from keytab as kudu/127.29.194.1@KRBTEST.COM (short username kudu)
I20260807 15:11:35.904778 1702 webserver.cc:460] Webserver started at http://127.29.194.1:37769/ using document root <none> and password file <none>
I20260807 15:11:35.905031 1702 fs_manager.cc:359] Metadata directory not provided
I20260807 15:11:35.905084 1702 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:11:35.905222 1702 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:11:35.906375 1702 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115480851507-30472-0/minicluster-data/ts-0/data/instance:
uuid: "1342a0971ddb4172883f3806ff83d922"
format_stamp: "Formatted at 2026-08-07 15:11:35 on dist-test-slave-42z9"
server_key: "f17210d6df9a0e7f31afa4232a230aef"
server_key_iv: "48251a066c65e2e06d4c1fabca08b0d3"
server_key_version: "encryptionkey@0"
I20260807 15:11:35.906762 1702 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115480851507-30472-0/minicluster-data/ts-0/wal/instance:
uuid: "1342a0971ddb4172883f3806ff83d922"
format_stamp: "Formatted at 2026-08-07 15:11:35 on dist-test-slave-42z9"
server_key: "f17210d6df9a0e7f31afa4232a230aef"
server_key_iv: "48251a066c65e2e06d4c1fabca08b0d3"
server_key_version: "encryptionkey@0"
I20260807 15:11:35.908141 1702 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260807 15:11:35.909015 1724 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:11:35.909195 1702 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260807 15:11:35.909305 1702 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115480851507-30472-0/minicluster-data/ts-0/data,/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115480851507-30472-0/minicluster-data/ts-0/wal
uuid: "1342a0971ddb4172883f3806ff83d922"
format_stamp: "Formatted at 2026-08-07 15:11:35 on dist-test-slave-42z9"
server_key: "f17210d6df9a0e7f31afa4232a230aef"
server_key_iv: "48251a066c65e2e06d4c1fabca08b0d3"
server_key_version: "encryptionkey@0"
I20260807 15:11:35.909407 1702 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115480851507-30472-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115480851507-30472-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115480851507-30472-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:11:35.940714 1702 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:11:35.943827 1702 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:11:35.944021 1702 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:11:35.944339 1702 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:11:35.944780 1702 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:11:35.944820 1702 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:11:35.944856 1702 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:11:35.944875 1702 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:11:35.951865 1702 rpc_server.cc:307] RPC server started. Bound to: 127.29.194.1:42741
I20260807 15:11:35.951913 1908 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.194.1:42741 every 8 connection(s)
I20260807 15:11:35.952270 1702 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115480851507-30472-0/minicluster-data/ts-0/data/info.pb
I20260807 15:11:35.955021 30472 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7fZSIL/build/release/bin/kudu as pid 1702
I20260807 15:11:35.955124 30472 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115480851507-30472-0/minicluster-data/ts-0/wal/instance
I20260807 15:11:35.955309 30472 external_mini_cluster.cc:1428] Setting key db583afcf5b024551b858e09000920c5
Aug 07 15:11:35 dist-test-slave-42z9 krb5kdc[1552](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115495, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.194.1@KRBTEST.COM for kudu/127.29.194.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.29.194.2@KRBTEST.COM; defaulting to no policy
I20260807 15:11:35.964403 1670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:35.953086 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.194.1:47503 (local address 127.29.194.62:39733)
0807 15:11:35.953175 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:35.953177 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:35.953407 (+ 230us) server_negotiation.cc:407] Connection header received
0807 15:11:35.953935 (+ 528us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:35.953937 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:35.953955 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:35.954052 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:11:35.955210 (+ 1158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:35.955844 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:35.957317 (+ 1473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:35.957611 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:35.957681 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:35.960897 (+ 3216us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:11:35.960899 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:11:35.960900 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:11:35.960915 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:11:35.962517 (+ 1602us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:35.963276 (+ 759us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:35.963279 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:35.963279 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:35.963374 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:35.963646 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:35.963647 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:35.963647 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:35.963774 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:11:35.963840 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0807 15:11:35.964178 (+ 338us) server_negotiation.cc:299] Negotiation successful
0807 15:11:35.964305 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260807 15:11:35.964556 1912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:35.953002 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:39733 (local address 127.29.194.1:47503)
0807 15:11:35.953176 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0807 15:11:35.953216 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0807 15:11:35.953775 (+ 559us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:11:35.954132 (+ 357us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:11:35.954136 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:11:35.954798 (+ 662us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:11:35.955030 (+ 232us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:35.955035 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:35.955978 (+ 943us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:35.955979 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:35.957176 (+ 1197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:35.957179 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:35.957725 (+ 546us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:35.957726 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:35.957858 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:35.958765 (+ 907us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:11:35.958775 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:11:35.960720 (+ 1945us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:11:35.962634 (+ 1914us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:11:35.962648 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:11:35.962652 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:11:35.963122 (+ 470us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:11:35.963449 (+ 327us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:11:35.963451 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:11:35.963451 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:11:35.963581 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:11:35.963838 (+ 257us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:11:35.963846 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:11:35.964039 (+ 193us) client_negotiation.cc:769] Sending connection context
0807 15:11:35.964142 (+ 103us) client_negotiation.cc:240] Negotiation successful
0807 15:11:35.964324 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":73,"threads_started":1}
I20260807 15:11:35.965095 1909 heartbeater.cc:347] Connected to a master server at 127.29.194.62:39733
I20260807 15:11:35.965198 1909 heartbeater.cc:464] Registering TS with master...
I20260807 15:11:35.965370 1909 heartbeater.cc:511] Master 127.29.194.62:39733 requested a full tablet report, sending...
I20260807 15:11:35.965997 1609 ts_manager.cc:194] Registered new tserver with Master: 1342a0971ddb4172883f3806ff83d922 (127.29.194.1:42741)
I20260807 15:11:35.966658 1609 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.194.1@KRBTEST.COM'} at 127.29.194.1:47503
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.194.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.194.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.2.keytab.
Entry for principal kudu/127.29.194.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.2.keytab.
I20260807 15:11:35.985939 30472 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7fZSIL/build/release/bin/kudu
/tmp/dist-test-task7fZSIL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115480851507-30472-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115480851507-30472-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115480851507-30472-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115480851507-30472-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.194.2:0
--local_ip_for_outbound_sockets=127.29.194.2
--webserver_interface=127.29.194.2
--webserver_port=0
--tserver_master_addrs=127.29.194.62:39733
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.2.keytab
--principal=kudu/127.29.194.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.194.20:37959
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115480851507-30472-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115480851507-30472-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115480851507-30472-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:11:36.078085 1917 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:11:36.078334 1917 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:11:36.078361 1917 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:11:36.079854 1917 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:11:36.079931 1917 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:11:36.079968 1917 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.194.2
I20260807 15:11:36.081737 1917 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.194.20:37959
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115480851507-30472-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115480851507-30472-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.2.keytab
--principal=kudu/127.29.194.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.194.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115480851507-30472-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.194.2
--webserver_port=0
--tserver_master_addrs=127.29.194.62:39733
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1917
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.194.2
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115480851507-30472-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:11:36.082001 1917 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:11:36.082281 1917 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:11:36.088373 1927 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:11:36.088573 1917 server_base.cc:1034] running on GCE node
W20260807 15:11:36.088528 1924 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:11:36.088528 1925 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:11:36.088940 1917 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:11:36.089249 1917 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:11:36.090442 1917 hybrid_clock.cc:648] HybridClock initialized: now 1786115496090413 us; error 79 us; skew 500 ppm
Aug 07 15:11:36 dist-test-slave-42z9 krb5kdc[1552](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115496, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.194.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:11:36.093204 1917 init.cc:377] Logged in from keytab as kudu/127.29.194.2@KRBTEST.COM (short username kudu)
I20260807 15:11:36.093736 1917 webserver.cc:460] Webserver started at http://127.29.194.2:40513/ using document root <none> and password file <none>
I20260807 15:11:36.093966 1917 fs_manager.cc:359] Metadata directory not provided
I20260807 15:11:36.094017 1917 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:11:36.094151 1917 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:11:36.095350 1917 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115480851507-30472-0/minicluster-data/ts-1/data/instance:
uuid: "0bf3660a67a5418bb3e2994f8431681c"
format_stamp: "Formatted at 2026-08-07 15:11:36 on dist-test-slave-42z9"
server_key: "651b015bff71a91e35cc5cf559825449"
server_key_iv: "511bcafc98cd8db10952c2e66b569ad3"
server_key_version: "encryptionkey@0"
I20260807 15:11:36.095710 1917 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115480851507-30472-0/minicluster-data/ts-1/wal/instance:
uuid: "0bf3660a67a5418bb3e2994f8431681c"
format_stamp: "Formatted at 2026-08-07 15:11:36 on dist-test-slave-42z9"
server_key: "651b015bff71a91e35cc5cf559825449"
server_key_iv: "511bcafc98cd8db10952c2e66b569ad3"
server_key_version: "encryptionkey@0"
I20260807 15:11:36.097105 1917 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 15:11:36.098528 1938 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:11:36.098863 1917 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 15:11:36.098989 1917 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115480851507-30472-0/minicluster-data/ts-1/data,/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115480851507-30472-0/minicluster-data/ts-1/wal
uuid: "0bf3660a67a5418bb3e2994f8431681c"
format_stamp: "Formatted at 2026-08-07 15:11:36 on dist-test-slave-42z9"
server_key: "651b015bff71a91e35cc5cf559825449"
server_key_iv: "511bcafc98cd8db10952c2e66b569ad3"
server_key_version: "encryptionkey@0"
I20260807 15:11:36.099073 1917 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115480851507-30472-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115480851507-30472-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115480851507-30472-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:11:36.142324 1917 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:11:36.144657 1917 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:11:36.144819 1917 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:11:36.145069 1917 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:11:36.145637 1917 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:11:36.145718 1917 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:11:36.145896 1917 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:11:36.145946 1917 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:11:36.153805 1917 rpc_server.cc:307] RPC server started. Bound to: 127.29.194.2:41827
I20260807 15:11:36.153868 2115 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.194.2:41827 every 8 connection(s)
I20260807 15:11:36.154207 1917 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115480851507-30472-0/minicluster-data/ts-1/data/info.pb
I20260807 15:11:36.154456 30472 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7fZSIL/build/release/bin/kudu as pid 1917
I20260807 15:11:36.154558 30472 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115480851507-30472-0/minicluster-data/ts-1/wal/instance
I20260807 15:11:36.154754 30472 external_mini_cluster.cc:1428] Setting key 4f312b71d55b83341fe676df73a87e63
Aug 07 15:11:36 dist-test-slave-42z9 krb5kdc[1552](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115496, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.194.2@KRBTEST.COM for kudu/127.29.194.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.29.194.3@KRBTEST.COM; defaulting to no policy
I20260807 15:11:36.164975 1670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:36.155069 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.194.2:49383 (local address 127.29.194.62:39733)
0807 15:11:36.155122 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:36.155124 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:36.155353 (+ 229us) server_negotiation.cc:407] Connection header received
0807 15:11:36.155909 (+ 556us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:36.155910 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:36.155923 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:36.156008 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:11:36.156965 (+ 957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:36.157768 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:36.158864 (+ 1096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:36.159135 (+ 271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:36.159209 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:36.161586 (+ 2377us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:11:36.161589 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:11:36.161590 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:11:36.161599 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:11:36.163324 (+ 1725us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:36.163793 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:36.163794 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:36.163795 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:36.163860 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:36.164181 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:36.164182 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:36.164183 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:36.164333 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:11:36.164395 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0807 15:11:36.164753 (+ 358us) server_negotiation.cc:299] Negotiation successful
0807 15:11:36.164901 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260807 15:11:36.165104 2118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:36.155014 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:39733 (local address 127.29.194.2:49383)
0807 15:11:36.155188 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0807 15:11:36.155246 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0807 15:11:36.155812 (+ 566us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:11:36.156084 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:11:36.156088 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:11:36.156668 (+ 580us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:11:36.156872 (+ 204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:36.156877 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:36.157901 (+ 1024us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:36.157902 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:36.158760 (+ 858us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:36.158763 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:36.159227 (+ 464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:36.159228 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:36.159360 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:36.159973 (+ 613us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:11:36.159982 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:11:36.161424 (+ 1442us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:11:36.163441 (+ 2017us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:11:36.163449 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:11:36.163451 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:11:36.163712 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:11:36.163971 (+ 259us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:11:36.163972 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:11:36.163972 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:11:36.164112 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:11:36.164425 (+ 313us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:11:36.164432 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:11:36.164579 (+ 147us) client_negotiation.cc:769] Sending connection context
0807 15:11:36.164685 (+ 106us) client_negotiation.cc:240] Negotiation successful
0807 15:11:36.164845 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":87,"threads_started":1}
I20260807 15:11:36.165637 2116 heartbeater.cc:347] Connected to a master server at 127.29.194.62:39733
I20260807 15:11:36.165741 2116 heartbeater.cc:464] Registering TS with master...
I20260807 15:11:36.165968 2116 heartbeater.cc:511] Master 127.29.194.62:39733 requested a full tablet report, sending...
I20260807 15:11:36.166699 1609 ts_manager.cc:194] Registered new tserver with Master: 0bf3660a67a5418bb3e2994f8431681c (127.29.194.2:41827)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.194.3@KRBTEST.COM" created.
I20260807 15:11:36.167207 1609 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.194.2@KRBTEST.COM'} at 127.29.194.2:49383
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.194.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.3.keytab.
Entry for principal kudu/127.29.194.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.3.keytab.
I20260807 15:11:36.185232 30472 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7fZSIL/build/release/bin/kudu
/tmp/dist-test-task7fZSIL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115480851507-30472-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115480851507-30472-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115480851507-30472-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115480851507-30472-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.194.3:0
--local_ip_for_outbound_sockets=127.29.194.3
--webserver_interface=127.29.194.3
--webserver_port=0
--tserver_master_addrs=127.29.194.62:39733
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.3.keytab
--principal=kudu/127.29.194.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.194.20:37959
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115480851507-30472-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115480851507-30472-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115480851507-30472-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:11:36.274940 2122 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:11:36.275166 2122 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:11:36.275190 2122 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:11:36.276394 2122 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:11:36.276446 2122 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:11:36.276475 2122 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.194.3
I20260807 15:11:36.278052 2122 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.194.20:37959
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115480851507-30472-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115480851507-30472-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.3.keytab
--principal=kudu/127.29.194.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.194.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115480851507-30472-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.194.3
--webserver_port=0
--tserver_master_addrs=127.29.194.62:39733
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2122
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.194.3
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115480851507-30472-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:11:36.278301 2122 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:11:36.278538 2122 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:11:36.284121 2130 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:11:36.284143 2128 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:11:36.284143 2127 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:11:36.284467 2122 server_base.cc:1034] running on GCE node
I20260807 15:11:36.284688 2122 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:11:36.284925 2122 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:11:36.286093 2122 hybrid_clock.cc:648] HybridClock initialized: now 1786115496286079 us; error 32 us; skew 500 ppm
Aug 07 15:11:36 dist-test-slave-42z9 krb5kdc[1552](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115496, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.194.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:11:36.289565 2122 init.cc:377] Logged in from keytab as kudu/127.29.194.3@KRBTEST.COM (short username kudu)
I20260807 15:11:36.290277 2122 webserver.cc:460] Webserver started at http://127.29.194.3:46705/ using document root <none> and password file <none>
I20260807 15:11:36.290529 2122 fs_manager.cc:359] Metadata directory not provided
I20260807 15:11:36.290585 2122 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:11:36.290723 2122 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:11:36.291854 2122 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115480851507-30472-0/minicluster-data/ts-2/data/instance:
uuid: "c867c3abb0bd4beeae84a068d8bd7a65"
format_stamp: "Formatted at 2026-08-07 15:11:36 on dist-test-slave-42z9"
server_key: "ca6292d7ae8d804ddad6c51346feb3f1"
server_key_iv: "a6d12bad602fc443d72840bd6f38a378"
server_key_version: "encryptionkey@0"
I20260807 15:11:36.292209 2122 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115480851507-30472-0/minicluster-data/ts-2/wal/instance:
uuid: "c867c3abb0bd4beeae84a068d8bd7a65"
format_stamp: "Formatted at 2026-08-07 15:11:36 on dist-test-slave-42z9"
server_key: "ca6292d7ae8d804ddad6c51346feb3f1"
server_key_iv: "a6d12bad602fc443d72840bd6f38a378"
server_key_version: "encryptionkey@0"
I20260807 15:11:36.293622 2122 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 15:11:36.294485 2141 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:11:36.294756 2122 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:11:36.294951 2122 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115480851507-30472-0/minicluster-data/ts-2/data,/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115480851507-30472-0/minicluster-data/ts-2/wal
uuid: "c867c3abb0bd4beeae84a068d8bd7a65"
format_stamp: "Formatted at 2026-08-07 15:11:36 on dist-test-slave-42z9"
server_key: "ca6292d7ae8d804ddad6c51346feb3f1"
server_key_iv: "a6d12bad602fc443d72840bd6f38a378"
server_key_version: "encryptionkey@0"
I20260807 15:11:36.295073 2122 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115480851507-30472-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115480851507-30472-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115480851507-30472-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:11:36.305030 2122 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:11:36.307859 2122 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:11:36.308014 2122 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:11:36.308260 2122 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:11:36.308566 2122 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:11:36.308598 2122 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:11:36.308619 2122 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:11:36.308630 2122 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:11:36.315727 2122 rpc_server.cc:307] RPC server started. Bound to: 127.29.194.3:36591
I20260807 15:11:36.315997 2305 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.194.3:36591 every 8 connection(s)
I20260807 15:11:36.316133 2122 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115480851507-30472-0/minicluster-data/ts-2/data/info.pb
I20260807 15:11:36.321867 30472 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7fZSIL/build/release/bin/kudu as pid 2122
I20260807 15:11:36.322037 30472 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115480851507-30472-0/minicluster-data/ts-2/wal/instance
I20260807 15:11:36.322348 30472 external_mini_cluster.cc:1428] Setting key e048b8fd84a7aa67f0fcef396cd499db
Aug 07 15:11:36 dist-test-slave-42z9 krb5kdc[1552](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115496, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.194.3@KRBTEST.COM for kudu/127.29.194.62@KRBTEST.COM
I20260807 15:11:36.326824 1670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:36.317052 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.194.3:57117 (local address 127.29.194.62:39733)
0807 15:11:36.317117 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:36.317119 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:36.317358 (+ 239us) server_negotiation.cc:407] Connection header received
0807 15:11:36.317862 (+ 504us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:36.317864 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:36.317875 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:36.317979 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:11:36.318963 (+ 984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:36.319498 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:36.320555 (+ 1057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:36.320752 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:36.320813 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:36.323860 (+ 3047us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:11:36.323861 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:11:36.323862 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:11:36.323870 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:11:36.325305 (+ 1435us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:36.325965 (+ 660us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:36.325966 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:36.325966 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:36.326034 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:36.326252 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:36.326252 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:36.326252 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:36.326363 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:11:36.326408 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0807 15:11:36.326677 (+ 269us) server_negotiation.cc:299] Negotiation successful
0807 15:11:36.326751 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260807 15:11:36.327020 2308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:36.317017 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:39733 (local address 127.29.194.3:57117)
0807 15:11:36.317186 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0807 15:11:36.317231 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0807 15:11:36.317774 (+ 543us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:11:36.317998 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:11:36.318001 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:11:36.318624 (+ 623us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:11:36.318833 (+ 209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:36.318838 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:36.319596 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:36.319597 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:36.320451 (+ 854us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:36.320455 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:36.320843 (+ 388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:36.320844 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:36.320963 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:36.321753 (+ 790us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:11:36.321770 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:11:36.323755 (+ 1985us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:11:36.325526 (+ 1771us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:11:36.325550 (+ 24us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:11:36.325553 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:11:36.325858 (+ 305us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:11:36.326089 (+ 231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:11:36.326090 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:11:36.326090 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:11:36.326199 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:11:36.326409 (+ 210us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:11:36.326412 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:11:36.326559 (+ 147us) client_negotiation.cc:769] Sending connection context
0807 15:11:36.326641 (+ 82us) client_negotiation.cc:240] Negotiation successful
0807 15:11:36.326789 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":91,"threads_started":1}
I20260807 15:11:36.327586 2306 heartbeater.cc:347] Connected to a master server at 127.29.194.62:39733
I20260807 15:11:36.327680 2306 heartbeater.cc:464] Registering TS with master...
I20260807 15:11:36.327845 2306 heartbeater.cc:511] Master 127.29.194.62:39733 requested a full tablet report, sending...
I20260807 15:11:36.328243 1609 ts_manager.cc:194] Registered new tserver with Master: c867c3abb0bd4beeae84a068d8bd7a65 (127.29.194.3:36591)
I20260807 15:11:36.328619 1609 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.194.3@KRBTEST.COM'} at 127.29.194.3:57117
I20260807 15:11:36.335201 30472 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260807 15:11:36.350129 1670 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:11:36.348784 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38452 (local address 127.29.194.62:39733)
0807 15:11:36.348929 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:36.348932 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:36.349075 (+ 143us) server_negotiation.cc:407] Connection header received
0807 15:11:36.349174 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:36.349175 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:36.349188 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:36.349247 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:11:36.350034 (+ 787us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38452: BlockingRecv error: recv got EOF from 127.0.0.1:38452 (error 108)
Metrics: {"server-negotiator.queue_time_us":83}
I20260807 15:11:36.351631 30472 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7fZSIL/build/release/bin/kudu with pid 1702
I20260807 15:11:36.366351 30472 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7fZSIL/build/release/bin/kudu with pid 1917
I20260807 15:11:36.386492 30472 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7fZSIL/build/release/bin/kudu with pid 2122
I20260807 15:11:36.403255 30472 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7fZSIL/build/release/bin/kudu with pid 1566
2026-08-07T15:11:36Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (2146 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786115480851507-30472-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 15:11:36 dist-test-slave-42z9 krb5kdc[2323](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 15:11:36 dist-test-slave-42z9 krb5kdc[2323](info): set up 2 sockets
Aug 07 15:11:36 dist-test-slave-42z9 krb5kdc[2323](info): commencing operation
krb5kdc: starting...
W20260807 15:11:37.579571 30472 mini_kdc.cc:121] Time spent starting KDC: real 1.142s user 0.000s sys 0.009s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786115480851507-30472-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786115480851507-30472-0/krb5kdc/test-user.keytab.
Aug 07 15:11:37 dist-test-slave-42z9 krb5kdc[2323](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115497, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T15:11:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T15:11:37Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.194.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.194.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.194.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.62.keytab.
Entry for principal kudu/127.29.194.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.62.keytab.
I20260807 15:11:37.721741 30472 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7fZSIL/build/release/bin/kudu
/tmp/dist-test-task7fZSIL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786115480851507-30472-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786115480851507-30472-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786115480851507-30472-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786115480851507-30472-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.194.62:33967
--webserver_interface=127.29.194.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.62.keytab
--principal=kudu/127.29.194.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.194.20:36457
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.194.62:33967
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786115480851507-30472-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786115480851507-30472-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786115480851507-30472-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:11:37.808377 2341 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:11:37.808605 2341 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:11:37.808640 2341 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:11:37.809948 2341 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:11:37.810001 2341 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:11:37.810021 2341 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:11:37.810036 2341 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:11:37.810052 2341 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:11:37.811729 2341 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.194.20:36457
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786115480851507-30472-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786115480851507-30472-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.194.62:33967
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.62.keytab
--principal=kudu/127.29.194.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.194.62:33967
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786115480851507-30472-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.194.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2341
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786115480851507-30472-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:11:37.811997 2341 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:11:37.812224 2341 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:11:37.817557 2349 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:11:37.817557 2352 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:11:37.817777 2350 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:11:37.817653 2341 server_base.cc:1034] running on GCE node
I20260807 15:11:37.818053 2341 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:11:37.818359 2341 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:11:37.819543 2341 hybrid_clock.cc:648] HybridClock initialized: now 1786115497819502 us; error 63 us; skew 500 ppm
Aug 07 15:11:37 dist-test-slave-42z9 krb5kdc[2323](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115497, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.194.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:11:37.827994 2341 init.cc:377] Logged in from keytab as kudu/127.29.194.62@KRBTEST.COM (short username kudu)
I20260807 15:11:37.828660 2341 webserver.cc:460] Webserver started at http://127.29.194.62:39953/ using document root <none> and password file <none>
I20260807 15:11:37.828935 2341 fs_manager.cc:359] Metadata directory not provided
I20260807 15:11:37.829041 2341 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:11:37.829221 2341 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:11:37.830430 2341 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786115480851507-30472-0/minicluster-data/master-0/data/instance:
uuid: "e803e33d766d4337b75c12ada9100a6b"
format_stamp: "Formatted at 2026-08-07 15:11:37 on dist-test-slave-42z9"
server_key: "26e14d0978abd112f7333cca652e575d"
server_key_iv: "7f7c6df61eb92fe1763d927521a07cdd"
server_key_version: "encryptionkey@0"
I20260807 15:11:37.830801 2341 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786115480851507-30472-0/minicluster-data/master-0/wal/instance:
uuid: "e803e33d766d4337b75c12ada9100a6b"
format_stamp: "Formatted at 2026-08-07 15:11:37 on dist-test-slave-42z9"
server_key: "26e14d0978abd112f7333cca652e575d"
server_key_iv: "7f7c6df61eb92fe1763d927521a07cdd"
server_key_version: "encryptionkey@0"
I20260807 15:11:37.832211 2341 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 15:11:37.833287 2361 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:11:37.833598 2341 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:11:37.833709 2341 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786115480851507-30472-0/minicluster-data/master-0/data,/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786115480851507-30472-0/minicluster-data/master-0/wal
uuid: "e803e33d766d4337b75c12ada9100a6b"
format_stamp: "Formatted at 2026-08-07 15:11:37 on dist-test-slave-42z9"
server_key: "26e14d0978abd112f7333cca652e575d"
server_key_iv: "7f7c6df61eb92fe1763d927521a07cdd"
server_key_version: "encryptionkey@0"
I20260807 15:11:37.833782 2341 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786115480851507-30472-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786115480851507-30472-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786115480851507-30472-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:11:37.857322 2341 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:11:37.860306 2341 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:11:37.860498 2341 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:11:37.865217 2341 rpc_server.cc:307] RPC server started. Bound to: 127.29.194.62:33967
I20260807 15:11:37.865684 2341 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786115480851507-30472-0/minicluster-data/master-0/data/info.pb
I20260807 15:11:37.865665 2438 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.194.62:33967 every 8 connection(s)
I20260807 15:11:37.866504 2440 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:11:37.868902 2440 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e803e33d766d4337b75c12ada9100a6b: Bootstrap starting.
I20260807 15:11:37.869112 30472 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7fZSIL/build/release/bin/kudu as pid 2341
I20260807 15:11:37.869212 30472 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786115480851507-30472-0/minicluster-data/master-0/wal/instance
I20260807 15:11:37.869408 30472 external_mini_cluster.cc:1428] Setting key 0ccb67235281fb38dd1916e04f047d77
I20260807 15:11:37.869722 2440 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e803e33d766d4337b75c12ada9100a6b: Neither blocks nor log segments found. Creating new log.
I20260807 15:11:37.870066 2440 log.cc:824] T 00000000000000000000000000000000 P e803e33d766d4337b75c12ada9100a6b: Log is configured to *not* fsync() on all Append() calls
I20260807 15:11:37.870808 2440 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e803e33d766d4337b75c12ada9100a6b: No bootstrap required, opened a new log
I20260807 15:11:37.872707 2440 raft_consensus.cc:348] T 00000000000000000000000000000000 P e803e33d766d4337b75c12ada9100a6b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e803e33d766d4337b75c12ada9100a6b" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 33967 } }
I20260807 15:11:37.872856 2440 raft_consensus.cc:374] T 00000000000000000000000000000000 P e803e33d766d4337b75c12ada9100a6b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:11:37.872879 2440 raft_consensus.cc:729] T 00000000000000000000000000000000 P e803e33d766d4337b75c12ada9100a6b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e803e33d766d4337b75c12ada9100a6b, State: Initialized, Role: FOLLOWER
I20260807 15:11:37.873008 2440 consensus_queue.cc:260] T 00000000000000000000000000000000 P e803e33d766d4337b75c12ada9100a6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e803e33d766d4337b75c12ada9100a6b" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 33967 } }
I20260807 15:11:37.873081 2440 raft_consensus.cc:388] T 00000000000000000000000000000000 P e803e33d766d4337b75c12ada9100a6b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:11:37.873104 2440 raft_consensus.cc:482] T 00000000000000000000000000000000 P e803e33d766d4337b75c12ada9100a6b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:11:37.873152 2440 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e803e33d766d4337b75c12ada9100a6b [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:11:37.873975 2440 raft_consensus.cc:504] T 00000000000000000000000000000000 P e803e33d766d4337b75c12ada9100a6b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e803e33d766d4337b75c12ada9100a6b" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 33967 } }
I20260807 15:11:37.874106 2440 leader_election.cc:302] T 00000000000000000000000000000000 P e803e33d766d4337b75c12ada9100a6b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e803e33d766d4337b75c12ada9100a6b; no voters:
I20260807 15:11:37.874281 2440 leader_election.cc:288] T 00000000000000000000000000000000 P e803e33d766d4337b75c12ada9100a6b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 15:11:37.874323 2445 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e803e33d766d4337b75c12ada9100a6b [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:11:37.874483 2445 raft_consensus.cc:686] T 00000000000000000000000000000000 P e803e33d766d4337b75c12ada9100a6b [term 1 LEADER]: Becoming Leader. State: Replica: e803e33d766d4337b75c12ada9100a6b, State: Running, Role: LEADER
I20260807 15:11:37.874665 2445 consensus_queue.cc:237] T 00000000000000000000000000000000 P e803e33d766d4337b75c12ada9100a6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e803e33d766d4337b75c12ada9100a6b" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 33967 } }
I20260807 15:11:37.874919 2440 sys_catalog.cc:564] T 00000000000000000000000000000000 P e803e33d766d4337b75c12ada9100a6b [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:11:37.875128 2446 sys_catalog.cc:455] T 00000000000000000000000000000000 P e803e33d766d4337b75c12ada9100a6b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e803e33d766d4337b75c12ada9100a6b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e803e33d766d4337b75c12ada9100a6b" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 33967 } } }
I20260807 15:11:37.875139 2447 sys_catalog.cc:455] T 00000000000000000000000000000000 P e803e33d766d4337b75c12ada9100a6b [sys.catalog]: SysCatalogTable state changed. Reason: New leader e803e33d766d4337b75c12ada9100a6b. Latest consensus state: current_term: 1 leader_uuid: "e803e33d766d4337b75c12ada9100a6b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e803e33d766d4337b75c12ada9100a6b" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 33967 } } }
I20260807 15:11:37.875229 2446 sys_catalog.cc:458] T 00000000000000000000000000000000 P e803e33d766d4337b75c12ada9100a6b [sys.catalog]: This master's current role is: LEADER
I20260807 15:11:37.875244 2447 sys_catalog.cc:458] T 00000000000000000000000000000000 P e803e33d766d4337b75c12ada9100a6b [sys.catalog]: This master's current role is: LEADER
I20260807 15:11:37.875702 2451 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:11:37.876288 2451 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 07 15:11:37 dist-test-slave-42z9 krb5kdc[2323](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115497, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.194.62@KRBTEST.COM
I20260807 15:11:37.878454 2451 catalog_manager.cc:1347] Generated new cluster ID: cab8cde955db4d6d977067f2fc6e02a9
I20260807 15:11:37.878588 2451 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:11:37.880582 2443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:37.869908 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51514 (local address 127.29.194.62:33967)
0807 15:11:37.870146 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:37.870157 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:37.870191 (+ 34us) server_negotiation.cc:407] Connection header received
0807 15:11:37.870746 (+ 555us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:37.870755 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:37.870807 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:37.870986 (+ 179us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:11:37.871804 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:37.872702 (+ 898us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:37.873653 (+ 951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:37.874085 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:37.874519 (+ 434us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:37.877136 (+ 2617us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:11:37.877141 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:11:37.877141 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:11:37.877155 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:11:37.879017 (+ 1862us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:37.879416 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:37.879419 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:37.879420 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:37.879520 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:37.879704 (+ 184us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:37.879704 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:37.879705 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:37.879953 (+ 248us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:11:37.880118 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0807 15:11:37.880211 (+ 93us) server_negotiation.cc:299] Negotiation successful
0807 15:11:37.880355 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":172,"thread_start_us":116,"threads_started":1}
I20260807 15:11:37.881546 30472 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 15:11:37.891546 2451 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:11:37.892304 2451 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:11:37.898864 2451 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e803e33d766d4337b75c12ada9100a6b: Generated new TSK 0
I20260807 15:11:37.899093 2451 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 15:11:37.900738 2443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:37.896381 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51526 (local address 127.29.194.62:33967)
0807 15:11:37.896432 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:37.896434 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:37.896555 (+ 121us) server_negotiation.cc:407] Connection header received
0807 15:11:37.896608 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:37.896609 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:37.896621 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:37.896700 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:11:37.897122 (+ 422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:37.898024 (+ 902us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:37.899092 (+ 1068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:37.899478 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:37.899551 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:37.900215 (+ 664us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:11:37.900216 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:11:37.900216 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:11:37.900457 (+ 241us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:11:37.900477 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:11:37.900485 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:11:37.900546 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0807 15:11:37.900622 (+ 76us) server_negotiation.cc:299] Negotiation successful
0807 15:11:37.900656 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260807 15:11:37.903946 30472 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7fZSIL/build/release/bin/kudu with pid 2341
2026-08-07T15:11:37Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1498 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115480851507-30472-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 15:11:37 dist-test-slave-42z9 krb5kdc[2484](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 15:11:37 dist-test-slave-42z9 krb5kdc[2484](info): set up 2 sockets
Aug 07 15:11:37 dist-test-slave-42z9 krb5kdc[2484](info): commencing operation
krb5kdc: starting...
W20260807 15:11:39.070125 30472 mini_kdc.cc:121] Time spent starting KDC: real 1.134s user 0.001s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115480851507-30472-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115480851507-30472-0/krb5kdc/test-user.keytab.
Aug 07 15:11:39 dist-test-slave-42z9 krb5kdc[2484](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115499, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T15:11:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T15:11:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.194.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.194.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.194.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.62.keytab.
Entry for principal kudu/127.29.194.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.62.keytab.
I20260807 15:11:39.212903 30472 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7fZSIL/build/release/bin/kudu
/tmp/dist-test-task7fZSIL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115480851507-30472-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115480851507-30472-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115480851507-30472-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115480851507-30472-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.194.62:40627
--webserver_interface=127.29.194.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.62.keytab
--principal=kudu/127.29.194.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.194.20:40633
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.194.62:40627
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115480851507-30472-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115480851507-30472-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115480851507-30472-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:11:39.299921 2499 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:11:39.300164 2499 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:11:39.300199 2499 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:11:39.301492 2499 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:11:39.301566 2499 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:11:39.301587 2499 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:11:39.301605 2499 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:11:39.301622 2499 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:11:39.303278 2499 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.194.20:40633
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115480851507-30472-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115480851507-30472-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.194.62:40627
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.62.keytab
--principal=kudu/127.29.194.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.194.62:40627
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115480851507-30472-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.194.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2499
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115480851507-30472-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:11:39.303532 2499 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:11:39.303732 2499 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:11:39.308825 2511 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:11:39.308825 2509 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:11:39.308779 2507 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:11:39.309291 2499 server_base.cc:1034] running on GCE node
I20260807 15:11:39.309765 2499 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:11:39.310091 2499 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:11:39.311293 2499 hybrid_clock.cc:648] HybridClock initialized: now 1786115499311262 us; error 51 us; skew 500 ppm
Aug 07 15:11:39 dist-test-slave-42z9 krb5kdc[2484](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115499, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.194.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:11:39.314224 2499 init.cc:377] Logged in from keytab as kudu/127.29.194.62@KRBTEST.COM (short username kudu)
I20260807 15:11:39.314739 2499 webserver.cc:460] Webserver started at http://127.29.194.62:40745/ using document root <none> and password file <none>
I20260807 15:11:39.314986 2499 fs_manager.cc:359] Metadata directory not provided
I20260807 15:11:39.315034 2499 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:11:39.315167 2499 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:11:39.316299 2499 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115480851507-30472-0/minicluster-data/master-0/data/instance:
uuid: "4cd323daa7c443df8c776a8097d493ee"
format_stamp: "Formatted at 2026-08-07 15:11:39 on dist-test-slave-42z9"
server_key: "e76aff852dc457b3f141ac3304648f4d"
server_key_iv: "4b9e56afd90974957505201464335cee"
server_key_version: "encryptionkey@0"
I20260807 15:11:39.316646 2499 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115480851507-30472-0/minicluster-data/master-0/wal/instance:
uuid: "4cd323daa7c443df8c776a8097d493ee"
format_stamp: "Formatted at 2026-08-07 15:11:39 on dist-test-slave-42z9"
server_key: "e76aff852dc457b3f141ac3304648f4d"
server_key_iv: "4b9e56afd90974957505201464335cee"
server_key_version: "encryptionkey@0"
I20260807 15:11:39.318135 2499 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 15:11:39.318977 2518 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:11:39.319168 2499 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:11:39.319273 2499 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115480851507-30472-0/minicluster-data/master-0/data,/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115480851507-30472-0/minicluster-data/master-0/wal
uuid: "4cd323daa7c443df8c776a8097d493ee"
format_stamp: "Formatted at 2026-08-07 15:11:39 on dist-test-slave-42z9"
server_key: "e76aff852dc457b3f141ac3304648f4d"
server_key_iv: "4b9e56afd90974957505201464335cee"
server_key_version: "encryptionkey@0"
I20260807 15:11:39.319358 2499 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115480851507-30472-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115480851507-30472-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115480851507-30472-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:11:39.339722 2499 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:11:39.342016 2499 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:11:39.342144 2499 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:11:39.346356 2499 rpc_server.cc:307] RPC server started. Bound to: 127.29.194.62:40627
I20260807 15:11:39.346510 2596 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.194.62:40627 every 8 connection(s)
I20260807 15:11:39.346801 2499 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115480851507-30472-0/minicluster-data/master-0/data/info.pb
I20260807 15:11:39.347762 2597 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:11:39.349347 30472 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7fZSIL/build/release/bin/kudu as pid 2499
I20260807 15:11:39.349468 30472 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115480851507-30472-0/minicluster-data/master-0/wal/instance
I20260807 15:11:39.349637 30472 external_mini_cluster.cc:1428] Setting key cd40d5af07ee7d99db6b86192e4ea567
I20260807 15:11:39.350646 2597 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4cd323daa7c443df8c776a8097d493ee: Bootstrap starting.
I20260807 15:11:39.351456 2597 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4cd323daa7c443df8c776a8097d493ee: Neither blocks nor log segments found. Creating new log.
I20260807 15:11:39.351749 2597 log.cc:824] T 00000000000000000000000000000000 P 4cd323daa7c443df8c776a8097d493ee: Log is configured to *not* fsync() on all Append() calls
I20260807 15:11:39.352540 2597 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4cd323daa7c443df8c776a8097d493ee: No bootstrap required, opened a new log
I20260807 15:11:39.354517 2597 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4cd323daa7c443df8c776a8097d493ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4cd323daa7c443df8c776a8097d493ee" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 40627 } }
I20260807 15:11:39.354676 2597 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4cd323daa7c443df8c776a8097d493ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:11:39.354708 2597 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4cd323daa7c443df8c776a8097d493ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4cd323daa7c443df8c776a8097d493ee, State: Initialized, Role: FOLLOWER
I20260807 15:11:39.354841 2597 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4cd323daa7c443df8c776a8097d493ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4cd323daa7c443df8c776a8097d493ee" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 40627 } }
I20260807 15:11:39.354933 2597 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4cd323daa7c443df8c776a8097d493ee [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:11:39.354982 2597 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4cd323daa7c443df8c776a8097d493ee [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:11:39.355032 2597 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4cd323daa7c443df8c776a8097d493ee [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:11:39.355669 2597 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4cd323daa7c443df8c776a8097d493ee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4cd323daa7c443df8c776a8097d493ee" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 40627 } }
I20260807 15:11:39.355773 2597 leader_election.cc:302] T 00000000000000000000000000000000 P 4cd323daa7c443df8c776a8097d493ee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4cd323daa7c443df8c776a8097d493ee; no voters:
I20260807 15:11:39.355970 2597 leader_election.cc:288] T 00000000000000000000000000000000 P 4cd323daa7c443df8c776a8097d493ee [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 15:11:39.355998 2602 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4cd323daa7c443df8c776a8097d493ee [term 1 FOLLOWER]: Leader election won for term 1
Aug 07 15:11:39 dist-test-slave-42z9 krb5kdc[2484](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115499, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.194.62@KRBTEST.COM
I20260807 15:11:39.356112 2602 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4cd323daa7c443df8c776a8097d493ee [term 1 LEADER]: Becoming Leader. State: Replica: 4cd323daa7c443df8c776a8097d493ee, State: Running, Role: LEADER
I20260807 15:11:39.356192 2602 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4cd323daa7c443df8c776a8097d493ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4cd323daa7c443df8c776a8097d493ee" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 40627 } }
I20260807 15:11:39.356258 2597 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4cd323daa7c443df8c776a8097d493ee [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:11:39.356575 2602 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4cd323daa7c443df8c776a8097d493ee [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4cd323daa7c443df8c776a8097d493ee. Latest consensus state: current_term: 1 leader_uuid: "4cd323daa7c443df8c776a8097d493ee" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4cd323daa7c443df8c776a8097d493ee" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 40627 } } }
I20260807 15:11:39.356642 2602 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4cd323daa7c443df8c776a8097d493ee [sys.catalog]: This master's current role is: LEADER
I20260807 15:11:39.356623 2604 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4cd323daa7c443df8c776a8097d493ee [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4cd323daa7c443df8c776a8097d493ee" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4cd323daa7c443df8c776a8097d493ee" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 40627 } } }
I20260807 15:11:39.356796 2604 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4cd323daa7c443df8c776a8097d493ee [sys.catalog]: This master's current role is: LEADER
I20260807 15:11:39.357088 2614 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:11:39.357853 2614 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:11:39.359900 2614 catalog_manager.cc:1347] Generated new cluster ID: 10e067dffcc340229a156e78dc5f6b74
I20260807 15:11:39.359956 2614 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:11:39.360407 2600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:39.350219 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58828 (local address 127.29.194.62:40627)
0807 15:11:39.350423 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:39.350432 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:39.350466 (+ 34us) server_negotiation.cc:407] Connection header received
0807 15:11:39.351031 (+ 565us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:39.351040 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:39.351109 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:39.351266 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:11:39.351924 (+ 658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:39.352897 (+ 973us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:39.353683 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:39.354098 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:39.354205 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:39.356788 (+ 2583us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:11:39.356793 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:11:39.356793 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:11:39.356806 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:11:39.358812 (+ 2006us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:39.359251 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:39.359254 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:39.359254 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:39.359382 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:39.359566 (+ 184us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:39.359567 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:39.359567 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:39.359808 (+ 241us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:11:39.359943 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0807 15:11:39.360064 (+ 121us) server_negotiation.cc:299] Negotiation successful
0807 15:11:39.360204 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":149,"thread_start_us":90,"threads_started":1}
I20260807 15:11:39.376258 2614 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:11:39.376832 2614 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:11:39.379976 2614 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4cd323daa7c443df8c776a8097d493ee: Generated new TSK 0
I20260807 15:11:39.380218 2614 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.29.194.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.194.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.194.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.1.keytab.
Entry for principal kudu/127.29.194.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.1.keytab.
I20260807 15:11:39.416324 30472 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7fZSIL/build/release/bin/kudu
/tmp/dist-test-task7fZSIL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115480851507-30472-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115480851507-30472-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115480851507-30472-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115480851507-30472-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.194.1:0
--local_ip_for_outbound_sockets=127.29.194.1
--webserver_interface=127.29.194.1
--webserver_port=0
--tserver_master_addrs=127.29.194.62:40627
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.1.keytab
--principal=kudu/127.29.194.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.194.20:40633
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115480851507-30472-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115480851507-30472-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115480851507-30472-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:11:39.508327 2636 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:11:39.508610 2636 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:11:39.508636 2636 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:11:39.509944 2636 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:11:39.509999 2636 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:11:39.510027 2636 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.194.1
I20260807 15:11:39.511647 2636 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.194.20:40633
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115480851507-30472-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115480851507-30472-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.1.keytab
--principal=kudu/127.29.194.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.194.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115480851507-30472-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.194.1
--webserver_port=0
--tserver_master_addrs=127.29.194.62:40627
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2636
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.194.1
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115480851507-30472-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:11:39.511917 2636 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:11:39.512187 2636 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:11:39.517809 2648 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:11:39.517896 2646 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:11:39.517930 2645 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:11:39.518203 2636 server_base.cc:1034] running on GCE node
I20260807 15:11:39.518414 2636 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:11:39.518646 2636 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:11:39.519819 2636 hybrid_clock.cc:648] HybridClock initialized: now 1786115499519772 us; error 85 us; skew 500 ppm
Aug 07 15:11:39 dist-test-slave-42z9 krb5kdc[2484](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115499, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.194.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:11:39.522599 2636 init.cc:377] Logged in from keytab as kudu/127.29.194.1@KRBTEST.COM (short username kudu)
I20260807 15:11:39.523068 2636 webserver.cc:460] Webserver started at http://127.29.194.1:36381/ using document root <none> and password file <none>
I20260807 15:11:39.523289 2636 fs_manager.cc:359] Metadata directory not provided
I20260807 15:11:39.523352 2636 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:11:39.523496 2636 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:11:39.524619 2636 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115480851507-30472-0/minicluster-data/ts-0/data/instance:
uuid: "c7f4307367534124ab9c73f1c544213a"
format_stamp: "Formatted at 2026-08-07 15:11:39 on dist-test-slave-42z9"
server_key: "a5b58fdfdfa885a618b0dc5e44e0cbc2"
server_key_iv: "51c12f056392040e4d84c3cbdf14145f"
server_key_version: "encryptionkey@0"
I20260807 15:11:39.524973 2636 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115480851507-30472-0/minicluster-data/ts-0/wal/instance:
uuid: "c7f4307367534124ab9c73f1c544213a"
format_stamp: "Formatted at 2026-08-07 15:11:39 on dist-test-slave-42z9"
server_key: "a5b58fdfdfa885a618b0dc5e44e0cbc2"
server_key_iv: "51c12f056392040e4d84c3cbdf14145f"
server_key_version: "encryptionkey@0"
I20260807 15:11:39.526477 2636 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260807 15:11:39.527441 2656 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:11:39.527717 2636 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 15:11:39.527806 2636 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115480851507-30472-0/minicluster-data/ts-0/data,/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115480851507-30472-0/minicluster-data/ts-0/wal
uuid: "c7f4307367534124ab9c73f1c544213a"
format_stamp: "Formatted at 2026-08-07 15:11:39 on dist-test-slave-42z9"
server_key: "a5b58fdfdfa885a618b0dc5e44e0cbc2"
server_key_iv: "51c12f056392040e4d84c3cbdf14145f"
server_key_version: "encryptionkey@0"
I20260807 15:11:39.527887 2636 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115480851507-30472-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115480851507-30472-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115480851507-30472-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:11:39.560704 2636 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:11:39.564592 2636 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:11:39.564811 2636 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:11:39.565163 2636 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:11:39.565716 2636 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:11:39.565773 2636 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:11:39.565802 2636 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:11:39.565814 2636 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:11:39.573540 2636 rpc_server.cc:307] RPC server started. Bound to: 127.29.194.1:34035
I20260807 15:11:39.573693 2819 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.194.1:34035 every 8 connection(s)
I20260807 15:11:39.574059 2636 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115480851507-30472-0/minicluster-data/ts-0/data/info.pb
Aug 07 15:11:39 dist-test-slave-42z9 krb5kdc[2484](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115499, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.194.1@KRBTEST.COM for kudu/127.29.194.62@KRBTEST.COM
I20260807 15:11:39.583423 30472 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7fZSIL/build/release/bin/kudu as pid 2636
I20260807 15:11:39.583567 30472 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115480851507-30472-0/minicluster-data/ts-0/wal/instance
I20260807 15:11:39.583752 30472 external_mini_cluster.cc:1428] Setting key 8f9fa5f5f582af8c329af6746ecae1e8
I20260807 15:11:39.585958 2600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:39.574986 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.194.1:41993 (local address 127.29.194.62:40627)
0807 15:11:39.575051 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:39.575053 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:39.575376 (+ 323us) server_negotiation.cc:407] Connection header received
0807 15:11:39.576048 (+ 672us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:39.576049 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:39.576062 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:39.576132 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:11:39.577491 (+ 1359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:39.578003 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:39.579134 (+ 1131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:39.579403 (+ 269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:39.579469 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:39.582598 (+ 3129us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:11:39.582600 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:11:39.582600 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:11:39.582609 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:11:39.584210 (+ 1601us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:39.584790 (+ 580us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:39.584791 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:39.584791 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:39.584864 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:39.585167 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:39.585168 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:39.585168 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:39.585329 (+ 161us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:11:39.585396 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0807 15:11:39.585764 (+ 368us) server_negotiation.cc:299] Negotiation successful
0807 15:11:39.585863 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260807 15:11:39.586094 2823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:39.574897 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:40627 (local address 127.29.194.1:41993)
0807 15:11:39.575095 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0807 15:11:39.575156 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0807 15:11:39.575807 (+ 651us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:11:39.576263 (+ 456us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:11:39.576267 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:11:39.576977 (+ 710us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:11:39.577204 (+ 227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:39.577210 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:39.578106 (+ 896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:39.578107 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:39.579014 (+ 907us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:39.579019 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:39.579512 (+ 493us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:39.579513 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:39.579640 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:39.580329 (+ 689us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:11:39.580342 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:11:39.582442 (+ 2100us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:11:39.584299 (+ 1857us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:11:39.584314 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:11:39.584317 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:11:39.584688 (+ 371us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:11:39.584965 (+ 277us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:11:39.584966 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:11:39.584966 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:11:39.585096 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:11:39.585423 (+ 327us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:11:39.585429 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:11:39.585626 (+ 197us) client_negotiation.cc:769] Sending connection context
0807 15:11:39.585722 (+ 96us) client_negotiation.cc:240] Negotiation successful
0807 15:11:39.585881 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":94,"threads_started":1}
I20260807 15:11:39.586661 2821 heartbeater.cc:347] Connected to a master server at 127.29.194.62:40627
I20260807 15:11:39.586776 2821 heartbeater.cc:464] Registering TS with master...
I20260807 15:11:39.586994 2821 heartbeater.cc:511] Master 127.29.194.62:40627 requested a full tablet report, sending...
I20260807 15:11:39.587707 2542 ts_manager.cc:194] Registered new tserver with Master: c7f4307367534124ab9c73f1c544213a (127.29.194.1:34035)
I20260807 15:11:39.588562 2542 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.194.1@KRBTEST.COM'} at 127.29.194.1:41993
WARNING: no policy specified for kudu/127.29.194.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.194.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.194.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.2.keytab.
Entry for principal kudu/127.29.194.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.2.keytab.
I20260807 15:11:39.613539 30472 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7fZSIL/build/release/bin/kudu
/tmp/dist-test-task7fZSIL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115480851507-30472-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115480851507-30472-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115480851507-30472-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115480851507-30472-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.194.2:0
--local_ip_for_outbound_sockets=127.29.194.2
--webserver_interface=127.29.194.2
--webserver_port=0
--tserver_master_addrs=127.29.194.62:40627
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.2.keytab
--principal=kudu/127.29.194.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.194.20:40633
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115480851507-30472-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115480851507-30472-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115480851507-30472-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:11:39.700255 2830 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:11:39.700579 2830 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:11:39.700613 2830 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:11:39.702883 2830 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:11:39.702965 2830 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:11:39.703004 2830 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.194.2
I20260807 15:11:39.705650 2830 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.194.20:40633
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115480851507-30472-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115480851507-30472-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.2.keytab
--principal=kudu/127.29.194.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.194.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115480851507-30472-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.194.2
--webserver_port=0
--tserver_master_addrs=127.29.194.62:40627
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2830
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.194.2
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115480851507-30472-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:11:39.705981 2830 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:11:39.706228 2830 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:11:39.711467 2839 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:11:39.711467 2840 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:11:39.711707 2830 server_base.cc:1034] running on GCE node
W20260807 15:11:39.711504 2844 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:11:39.711962 2830 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:11:39.712239 2830 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:11:39.713410 2830 hybrid_clock.cc:648] HybridClock initialized: now 1786115499713390 us; error 36 us; skew 500 ppm
Aug 07 15:11:39 dist-test-slave-42z9 krb5kdc[2484](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115499, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.194.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:11:39.716522 2830 init.cc:377] Logged in from keytab as kudu/127.29.194.2@KRBTEST.COM (short username kudu)
I20260807 15:11:39.717170 2830 webserver.cc:460] Webserver started at http://127.29.194.2:46353/ using document root <none> and password file <none>
I20260807 15:11:39.717502 2830 fs_manager.cc:359] Metadata directory not provided
I20260807 15:11:39.717566 2830 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:11:39.717862 2830 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:11:39.719096 2830 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115480851507-30472-0/minicluster-data/ts-1/data/instance:
uuid: "1c99993936c44616bada7d460aad1572"
format_stamp: "Formatted at 2026-08-07 15:11:39 on dist-test-slave-42z9"
server_key: "4aeebb70e427643da3de78df943d7829"
server_key_iv: "bcb2239d3a752affd77c2cae8e991fcb"
server_key_version: "encryptionkey@0"
I20260807 15:11:39.719515 2830 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115480851507-30472-0/minicluster-data/ts-1/wal/instance:
uuid: "1c99993936c44616bada7d460aad1572"
format_stamp: "Formatted at 2026-08-07 15:11:39 on dist-test-slave-42z9"
server_key: "4aeebb70e427643da3de78df943d7829"
server_key_iv: "bcb2239d3a752affd77c2cae8e991fcb"
server_key_version: "encryptionkey@0"
I20260807 15:11:39.721060 2830 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260807 15:11:39.722007 2853 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:11:39.722219 2830 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260807 15:11:39.722299 2830 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115480851507-30472-0/minicluster-data/ts-1/data,/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115480851507-30472-0/minicluster-data/ts-1/wal
uuid: "1c99993936c44616bada7d460aad1572"
format_stamp: "Formatted at 2026-08-07 15:11:39 on dist-test-slave-42z9"
server_key: "4aeebb70e427643da3de78df943d7829"
server_key_iv: "bcb2239d3a752affd77c2cae8e991fcb"
server_key_version: "encryptionkey@0"
I20260807 15:11:39.722379 2830 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115480851507-30472-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115480851507-30472-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115480851507-30472-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:11:39.749431 2830 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:11:39.751690 2830 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:11:39.751816 2830 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:11:39.752084 2830 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:11:39.752411 2830 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:11:39.752452 2830 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:11:39.752476 2830 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:11:39.752487 2830 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:11:39.759522 2830 rpc_server.cc:307] RPC server started. Bound to: 127.29.194.2:38619
I20260807 15:11:39.759579 3017 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.194.2:38619 every 8 connection(s)
I20260807 15:11:39.759925 2830 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115480851507-30472-0/minicluster-data/ts-1/data/info.pb
I20260807 15:11:39.760282 30472 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7fZSIL/build/release/bin/kudu as pid 2830
I20260807 15:11:39.760378 30472 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115480851507-30472-0/minicluster-data/ts-1/wal/instance
I20260807 15:11:39.760555 30472 external_mini_cluster.cc:1428] Setting key 60c4915ace0d4e1789f452f5be175203
Aug 07 15:11:39 dist-test-slave-42z9 krb5kdc[2484](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115499, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.194.2@KRBTEST.COM for kudu/127.29.194.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.29.194.3@KRBTEST.COM; defaulting to no policy
I20260807 15:11:39.772315 2600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:39.761267 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.194.2:55033 (local address 127.29.194.62:40627)
0807 15:11:39.761350 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:39.761353 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:39.761708 (+ 355us) server_negotiation.cc:407] Connection header received
0807 15:11:39.762341 (+ 633us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:39.762343 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:39.762358 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:39.762470 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:11:39.763550 (+ 1080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:39.764291 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:39.765741 (+ 1450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:39.766049 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:39.766122 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:39.768801 (+ 2679us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:11:39.768803 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:11:39.768803 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:11:39.768811 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:11:39.770619 (+ 1808us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:39.771124 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:39.771125 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:39.771125 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:39.771189 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:39.771534 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:39.771535 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:39.771536 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:39.771671 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:11:39.771779 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0807 15:11:39.772105 (+ 326us) server_negotiation.cc:299] Negotiation successful
0807 15:11:39.772209 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260807 15:11:39.772488 3022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:39.761233 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:40627 (local address 127.29.194.2:55033)
0807 15:11:39.761491 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0807 15:11:39.761563 (+ 72us) client_negotiation.cc:174] Beginning negotiation
0807 15:11:39.762230 (+ 667us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:11:39.762584 (+ 354us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:11:39.762589 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:11:39.763209 (+ 620us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:11:39.763438 (+ 229us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:39.763444 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:39.764466 (+ 1022us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:39.764468 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:39.765537 (+ 1069us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:39.765542 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:39.766158 (+ 616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:39.766159 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:39.766320 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:39.766973 (+ 653us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:11:39.766982 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:11:39.768649 (+ 1667us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:11:39.770761 (+ 2112us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:11:39.770770 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:11:39.770773 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:11:39.771027 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:11:39.771295 (+ 268us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:11:39.771296 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:11:39.771297 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:11:39.771422 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:11:39.771766 (+ 344us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:11:39.771774 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:11:39.771928 (+ 154us) client_negotiation.cc:769] Sending connection context
0807 15:11:39.772050 (+ 122us) client_negotiation.cc:240] Negotiation successful
0807 15:11:39.772246 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":88,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.194.3@KRBTEST.COM" created.
I20260807 15:11:39.773108 3020 heartbeater.cc:347] Connected to a master server at 127.29.194.62:40627
I20260807 15:11:39.773217 3020 heartbeater.cc:464] Registering TS with master...
I20260807 15:11:39.773414 3020 heartbeater.cc:511] Master 127.29.194.62:40627 requested a full tablet report, sending...
I20260807 15:11:39.773881 2542 ts_manager.cc:194] Registered new tserver with Master: 1c99993936c44616bada7d460aad1572 (127.29.194.2:38619)
I20260807 15:11:39.774263 2542 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.194.2@KRBTEST.COM'} at 127.29.194.2:55033
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.194.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.3.keytab.
Entry for principal kudu/127.29.194.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.3.keytab.
I20260807 15:11:39.793272 30472 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7fZSIL/build/release/bin/kudu
/tmp/dist-test-task7fZSIL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115480851507-30472-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115480851507-30472-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115480851507-30472-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115480851507-30472-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.194.3:0
--local_ip_for_outbound_sockets=127.29.194.3
--webserver_interface=127.29.194.3
--webserver_port=0
--tserver_master_addrs=127.29.194.62:40627
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.3.keytab
--principal=kudu/127.29.194.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.194.20:40633
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115480851507-30472-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115480851507-30472-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115480851507-30472-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:11:39.882054 3025 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:11:39.882302 3025 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:11:39.882329 3025 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:11:39.884325 3025 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:11:39.884426 3025 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:11:39.884462 3025 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.194.3
I20260807 15:11:39.886435 3025 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.194.20:40633
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115480851507-30472-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115480851507-30472-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.3.keytab
--principal=kudu/127.29.194.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.194.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115480851507-30472-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.194.3
--webserver_port=0
--tserver_master_addrs=127.29.194.62:40627
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3025
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.194.3
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115480851507-30472-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:11:39.886700 3025 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:11:39.886905 3025 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:11:39.892387 3032 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:11:39.892531 3025 server_base.cc:1034] running on GCE node
W20260807 15:11:39.892467 3033 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:11:39.892395 3036 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:11:39.892938 3025 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:11:39.893220 3025 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:11:39.894393 3025 hybrid_clock.cc:648] HybridClock initialized: now 1786115499894339 us; error 67 us; skew 500 ppm
Aug 07 15:11:39 dist-test-slave-42z9 krb5kdc[2484](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115499, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.194.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:11:39.897820 3025 init.cc:377] Logged in from keytab as kudu/127.29.194.3@KRBTEST.COM (short username kudu)
I20260807 15:11:39.898496 3025 webserver.cc:460] Webserver started at http://127.29.194.3:45343/ using document root <none> and password file <none>
I20260807 15:11:39.898747 3025 fs_manager.cc:359] Metadata directory not provided
I20260807 15:11:39.899000 3025 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:11:39.899178 3025 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:11:39.900383 3025 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115480851507-30472-0/minicluster-data/ts-2/data/instance:
uuid: "9b3420a441da45ca8039eadc928f8a7f"
format_stamp: "Formatted at 2026-08-07 15:11:39 on dist-test-slave-42z9"
server_key: "50789c316b28cf82e3ed3d3d0b6f3417"
server_key_iv: "274c0f1cc7191826c404cbf704a5e82f"
server_key_version: "encryptionkey@0"
I20260807 15:11:39.900714 3025 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115480851507-30472-0/minicluster-data/ts-2/wal/instance:
uuid: "9b3420a441da45ca8039eadc928f8a7f"
format_stamp: "Formatted at 2026-08-07 15:11:39 on dist-test-slave-42z9"
server_key: "50789c316b28cf82e3ed3d3d0b6f3417"
server_key_iv: "274c0f1cc7191826c404cbf704a5e82f"
server_key_version: "encryptionkey@0"
I20260807 15:11:39.902290 3025 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 15:11:39.903517 3048 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:11:39.903826 3025 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:11:39.903970 3025 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115480851507-30472-0/minicluster-data/ts-2/data,/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115480851507-30472-0/minicluster-data/ts-2/wal
uuid: "9b3420a441da45ca8039eadc928f8a7f"
format_stamp: "Formatted at 2026-08-07 15:11:39 on dist-test-slave-42z9"
server_key: "50789c316b28cf82e3ed3d3d0b6f3417"
server_key_iv: "274c0f1cc7191826c404cbf704a5e82f"
server_key_version: "encryptionkey@0"
I20260807 15:11:39.904057 3025 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115480851507-30472-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115480851507-30472-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115480851507-30472-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:11:39.917420 3025 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:11:39.920099 3025 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:11:39.920271 3025 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:11:39.920552 3025 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:11:39.920871 3025 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:11:39.920905 3025 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:11:39.920936 3025 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:11:39.920962 3025 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:11:39.927211 3025 rpc_server.cc:307] RPC server started. Bound to: 127.29.194.3:42461
I20260807 15:11:39.927282 3209 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.194.3:42461 every 8 connection(s)
I20260807 15:11:39.927597 3025 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115480851507-30472-0/minicluster-data/ts-2/data/info.pb
I20260807 15:11:39.930033 30472 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7fZSIL/build/release/bin/kudu as pid 3025
I20260807 15:11:39.930106 30472 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115480851507-30472-0/minicluster-data/ts-2/wal/instance
I20260807 15:11:39.930294 30472 external_mini_cluster.cc:1428] Setting key 7a52b61b4102e5a8c9c7171721451e3d
Aug 07 15:11:39 dist-test-slave-42z9 krb5kdc[2484](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115499, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.194.3@KRBTEST.COM for kudu/127.29.194.62@KRBTEST.COM
I20260807 15:11:39.937942 2600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:39.928328 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.194.3:36487 (local address 127.29.194.62:40627)
0807 15:11:39.928382 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:39.928384 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:39.928595 (+ 211us) server_negotiation.cc:407] Connection header received
0807 15:11:39.929080 (+ 485us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:39.929081 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:39.929091 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:39.929141 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:11:39.930211 (+ 1070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:39.930889 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:39.932026 (+ 1137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:39.932235 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:39.932303 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:39.934765 (+ 2462us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:11:39.934767 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:11:39.934767 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:11:39.934775 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:11:39.936126 (+ 1351us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:39.936650 (+ 524us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:39.936651 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:39.936651 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:39.936717 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:39.937234 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:39.937235 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:39.937235 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:39.937370 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:11:39.937517 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0807 15:11:39.937766 (+ 249us) server_negotiation.cc:299] Negotiation successful
0807 15:11:39.937869 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260807 15:11:39.938071 3214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:39.928317 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:40627 (local address 127.29.194.3:36487)
0807 15:11:39.928469 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0807 15:11:39.928505 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0807 15:11:39.929000 (+ 495us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:11:39.929197 (+ 197us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:11:39.929200 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:11:39.929837 (+ 637us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:11:39.930117 (+ 280us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:39.930125 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:39.930981 (+ 856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:39.930981 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:39.931905 (+ 924us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:39.931908 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:39.932373 (+ 465us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:39.932374 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:39.932501 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:39.933031 (+ 530us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:11:39.933040 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:11:39.934637 (+ 1597us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:11:39.936239 (+ 1602us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:11:39.936247 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:11:39.936250 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:11:39.936558 (+ 308us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:11:39.936958 (+ 400us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:11:39.936959 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:11:39.936960 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:11:39.937099 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:11:39.937498 (+ 399us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:11:39.937506 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:11:39.937639 (+ 133us) client_negotiation.cc:769] Sending connection context
0807 15:11:39.937733 (+ 94us) client_negotiation.cc:240] Negotiation successful
0807 15:11:39.937875 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":86,"threads_started":1}
I20260807 15:11:39.938791 3210 heartbeater.cc:347] Connected to a master server at 127.29.194.62:40627
I20260807 15:11:39.938900 3210 heartbeater.cc:464] Registering TS with master...
I20260807 15:11:39.939102 3210 heartbeater.cc:511] Master 127.29.194.62:40627 requested a full tablet report, sending...
I20260807 15:11:39.939628 2542 ts_manager.cc:194] Registered new tserver with Master: 9b3420a441da45ca8039eadc928f8a7f (127.29.194.3:42461)
I20260807 15:11:39.940011 2542 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.194.3@KRBTEST.COM'} at 127.29.194.3:36487
I20260807 15:11:39.942407 30472 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 07 15:11:39 dist-test-slave-42z9 krb5kdc[2484](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115499, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 07 15:11:39 dist-test-slave-42z9 krb5kdc[2484](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115499, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.29.194.62@KRBTEST.COM
I20260807 15:11:39.970849 2600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:39.963125 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49552 (local address 127.29.194.62:40627)
0807 15:11:39.963228 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:39.963230 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:39.963245 (+ 15us) server_negotiation.cc:407] Connection header received
0807 15:11:39.963396 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:39.963399 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:39.963418 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:39.963513 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:11:39.964189 (+ 676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:39.964736 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:39.965602 (+ 866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:39.965805 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:39.965878 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:39.968205 (+ 2327us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:11:39.968207 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:11:39.968207 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:11:39.968215 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:11:39.969636 (+ 1421us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:39.970012 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:39.970013 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:39.970014 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:39.970079 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:39.970309 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:39.970309 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:39.970310 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:39.970490 (+ 180us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:11:39.970569 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0807 15:11:39.970707 (+ 138us) server_negotiation.cc:299] Negotiation successful
0807 15:11:39.970779 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
W20260807 15:11:39.971163 2542 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:49552
I20260807 15:11:39.972003 30472 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7fZSIL/build/release/bin/kudu with pid 2636
I20260807 15:11:40.014061 30472 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7fZSIL/build/release/bin/kudu with pid 2830
I20260807 15:11:40.033741 30472 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7fZSIL/build/release/bin/kudu with pid 3025
I20260807 15:11:40.048055 30472 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7fZSIL/build/release/bin/kudu with pid 2499
2026-08-07T15:11:40Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (2146 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115480851507-30472-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 15:11:40 dist-test-slave-42z9 krb5kdc[3228](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 15:11:40 dist-test-slave-42z9 krb5kdc[3228](info): set up 2 sockets
Aug 07 15:11:40 dist-test-slave-42z9 krb5kdc[3228](info): commencing operation
krb5kdc: starting...
W20260807 15:11:41.236400 30472 mini_kdc.cc:121] Time spent starting KDC: real 1.154s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115480851507-30472-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115480851507-30472-0/krb5kdc/test-user.keytab.
Aug 07 15:11:41 dist-test-slave-42z9 krb5kdc[3228](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115501, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T15:11:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T15:11:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.194.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.194.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.194.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.62.keytab.
Entry for principal kudu/127.29.194.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.62.keytab.
I20260807 15:11:41.381923 30472 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7fZSIL/build/release/bin/kudu
/tmp/dist-test-task7fZSIL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115480851507-30472-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115480851507-30472-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115480851507-30472-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115480851507-30472-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.194.62:36595
--webserver_interface=127.29.194.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.62.keytab
--principal=kudu/127.29.194.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.194.20:34113
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.194.62:36595
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115480851507-30472-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115480851507-30472-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115480851507-30472-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:11:41.484802 3246 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:11:41.485123 3246 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:11:41.485280 3246 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:11:41.488076 3246 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:11:41.488190 3246 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:11:41.488216 3246 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:11:41.488238 3246 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:11:41.488256 3246 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:11:41.490566 3246 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.194.20:34113
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115480851507-30472-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115480851507-30472-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.194.62:36595
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.62.keytab
--principal=kudu/127.29.194.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.194.62:36595
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115480851507-30472-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.194.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3246
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115480851507-30472-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:11:41.490912 3246 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:11:41.491200 3246 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:11:41.498164 3252 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:11:41.498411 3246 server_base.cc:1034] running on GCE node
W20260807 15:11:41.498167 3253 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:11:41.498164 3256 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:11:41.498925 3246 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:11:41.499449 3246 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:11:41.500653 3246 hybrid_clock.cc:648] HybridClock initialized: now 1786115501500637 us; error 38 us; skew 500 ppm
Aug 07 15:11:41 dist-test-slave-42z9 krb5kdc[3228](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115501, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.194.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:11:41.504072 3246 init.cc:377] Logged in from keytab as kudu/127.29.194.62@KRBTEST.COM (short username kudu)
I20260807 15:11:41.504649 3246 webserver.cc:460] Webserver started at http://127.29.194.62:33207/ using document root <none> and password file <none>
I20260807 15:11:41.504921 3246 fs_manager.cc:359] Metadata directory not provided
I20260807 15:11:41.504976 3246 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:11:41.505115 3246 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:11:41.506327 3246 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115480851507-30472-0/minicluster-data/master-0/data/instance:
uuid: "ae231238688841a189ba48594c5e791c"
format_stamp: "Formatted at 2026-08-07 15:11:41 on dist-test-slave-42z9"
server_key: "c0248524e9c92407008616501a23c43d"
server_key_iv: "53023307c7161a907d558d197b7e2129"
server_key_version: "encryptionkey@0"
I20260807 15:11:41.506688 3246 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115480851507-30472-0/minicluster-data/master-0/wal/instance:
uuid: "ae231238688841a189ba48594c5e791c"
format_stamp: "Formatted at 2026-08-07 15:11:41 on dist-test-slave-42z9"
server_key: "c0248524e9c92407008616501a23c43d"
server_key_iv: "53023307c7161a907d558d197b7e2129"
server_key_version: "encryptionkey@0"
I20260807 15:11:41.508416 3246 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.002s
I20260807 15:11:41.509493 3270 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:11:41.509848 3246 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:11:41.509970 3246 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115480851507-30472-0/minicluster-data/master-0/data,/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115480851507-30472-0/minicluster-data/master-0/wal
uuid: "ae231238688841a189ba48594c5e791c"
format_stamp: "Formatted at 2026-08-07 15:11:41 on dist-test-slave-42z9"
server_key: "c0248524e9c92407008616501a23c43d"
server_key_iv: "53023307c7161a907d558d197b7e2129"
server_key_version: "encryptionkey@0"
I20260807 15:11:41.510051 3246 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115480851507-30472-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115480851507-30472-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115480851507-30472-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:11:41.554030 3246 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:11:41.558616 3246 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:11:41.558888 3246 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:11:41.566685 3246 rpc_server.cc:307] RPC server started. Bound to: 127.29.194.62:36595
I20260807 15:11:41.566787 3339 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.194.62:36595 every 8 connection(s)
I20260807 15:11:41.567426 3246 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115480851507-30472-0/minicluster-data/master-0/data/info.pb
I20260807 15:11:41.568246 3340 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:11:41.570976 30472 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7fZSIL/build/release/bin/kudu as pid 3246
I20260807 15:11:41.571044 3340 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ae231238688841a189ba48594c5e791c: Bootstrap starting.
I20260807 15:11:41.571096 30472 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115480851507-30472-0/minicluster-data/master-0/wal/instance
I20260807 15:11:41.571256 30472 external_mini_cluster.cc:1428] Setting key ea0eaf0ec3e30e2d2aac3c7a3009ee17
I20260807 15:11:41.571774 3340 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ae231238688841a189ba48594c5e791c: Neither blocks nor log segments found. Creating new log.
I20260807 15:11:41.572141 3340 log.cc:824] T 00000000000000000000000000000000 P ae231238688841a189ba48594c5e791c: Log is configured to *not* fsync() on all Append() calls
I20260807 15:11:41.573122 3340 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ae231238688841a189ba48594c5e791c: No bootstrap required, opened a new log
I20260807 15:11:41.575809 3340 raft_consensus.cc:348] T 00000000000000000000000000000000 P ae231238688841a189ba48594c5e791c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae231238688841a189ba48594c5e791c" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 36595 } }
I20260807 15:11:41.576083 3340 raft_consensus.cc:374] T 00000000000000000000000000000000 P ae231238688841a189ba48594c5e791c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:11:41.576110 3340 raft_consensus.cc:729] T 00000000000000000000000000000000 P ae231238688841a189ba48594c5e791c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ae231238688841a189ba48594c5e791c, State: Initialized, Role: FOLLOWER
I20260807 15:11:41.576274 3340 consensus_queue.cc:260] T 00000000000000000000000000000000 P ae231238688841a189ba48594c5e791c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae231238688841a189ba48594c5e791c" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 36595 } }
I20260807 15:11:41.576607 3340 raft_consensus.cc:388] T 00000000000000000000000000000000 P ae231238688841a189ba48594c5e791c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:11:41.576735 3340 raft_consensus.cc:482] T 00000000000000000000000000000000 P ae231238688841a189ba48594c5e791c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:11:41.576797 3340 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ae231238688841a189ba48594c5e791c [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:11:41.577685 3340 raft_consensus.cc:504] T 00000000000000000000000000000000 P ae231238688841a189ba48594c5e791c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae231238688841a189ba48594c5e791c" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 36595 } }
I20260807 15:11:41.577920 3340 leader_election.cc:302] T 00000000000000000000000000000000 P ae231238688841a189ba48594c5e791c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ae231238688841a189ba48594c5e791c; no voters:
I20260807 15:11:41.578155 3340 leader_election.cc:288] T 00000000000000000000000000000000 P ae231238688841a189ba48594c5e791c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 15:11:41.578198 3345 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ae231238688841a189ba48594c5e791c [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:11:41.578362 3345 raft_consensus.cc:686] T 00000000000000000000000000000000 P ae231238688841a189ba48594c5e791c [term 1 LEADER]: Becoming Leader. State: Replica: ae231238688841a189ba48594c5e791c, State: Running, Role: LEADER
I20260807 15:11:41.578495 3340 sys_catalog.cc:564] T 00000000000000000000000000000000 P ae231238688841a189ba48594c5e791c [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:11:41.578553 3345 consensus_queue.cc:237] T 00000000000000000000000000000000 P ae231238688841a189ba48594c5e791c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae231238688841a189ba48594c5e791c" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 36595 } }
I20260807 15:11:41.579538 3345 sys_catalog.cc:455] T 00000000000000000000000000000000 P ae231238688841a189ba48594c5e791c [sys.catalog]: SysCatalogTable state changed. Reason: New leader ae231238688841a189ba48594c5e791c. Latest consensus state: current_term: 1 leader_uuid: "ae231238688841a189ba48594c5e791c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae231238688841a189ba48594c5e791c" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 36595 } } }
I20260807 15:11:41.579658 3345 sys_catalog.cc:458] T 00000000000000000000000000000000 P ae231238688841a189ba48594c5e791c [sys.catalog]: This master's current role is: LEADER
I20260807 15:11:41.579762 3347 sys_catalog.cc:455] T 00000000000000000000000000000000 P ae231238688841a189ba48594c5e791c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ae231238688841a189ba48594c5e791c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae231238688841a189ba48594c5e791c" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 36595 } } }
I20260807 15:11:41.579842 3347 sys_catalog.cc:458] T 00000000000000000000000000000000 P ae231238688841a189ba48594c5e791c [sys.catalog]: This master's current role is: LEADER
I20260807 15:11:41.579972 3360 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:11:41.580632 3360 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 07 15:11:41 dist-test-slave-42z9 krb5kdc[3228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115501, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.194.62@KRBTEST.COM
I20260807 15:11:41.583611 3360 catalog_manager.cc:1347] Generated new cluster ID: 1e2170fcdf1549daae538f049e590a82
I20260807 15:11:41.583683 3360 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:11:41.588733 3343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:41.571865 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41064 (local address 127.29.194.62:36595)
0807 15:11:41.572316 (+ 451us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:41.572327 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:41.572370 (+ 43us) server_negotiation.cc:407] Connection header received
0807 15:11:41.573424 (+ 1054us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:41.573433 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:41.573583 (+ 150us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:41.573837 (+ 254us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:11:41.574728 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:41.576557 (+ 1829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:41.577672 (+ 1115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:41.578475 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:41.579059 (+ 584us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:41.582169 (+ 3110us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:11:41.582174 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:11:41.582174 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:11:41.582190 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:11:41.586629 (+ 4439us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:41.587251 (+ 622us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:41.587255 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:41.587255 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:41.587372 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:41.587589 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:41.587589 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:41.587590 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:41.587911 (+ 321us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:11:41.588071 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0807 15:11:41.588199 (+ 128us) server_negotiation.cc:299] Negotiation successful
0807 15:11:41.588473 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":373,"thread_start_us":112,"threads_started":1}
I20260807 15:11:41.598475 3360 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:11:41.599575 3360 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:11:41.603726 3360 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ae231238688841a189ba48594c5e791c: Generated new TSK 0
I20260807 15:11:41.603958 3360 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.29.194.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.194.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.194.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.1.keytab.
Entry for principal kudu/127.29.194.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.1.keytab.
I20260807 15:11:41.645732 30472 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7fZSIL/build/release/bin/kudu
/tmp/dist-test-task7fZSIL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115480851507-30472-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115480851507-30472-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115480851507-30472-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115480851507-30472-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.194.1:0
--local_ip_for_outbound_sockets=127.29.194.1
--webserver_interface=127.29.194.1
--webserver_port=0
--tserver_master_addrs=127.29.194.62:36595
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.1.keytab
--principal=kudu/127.29.194.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.194.20:34113
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115480851507-30472-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115480851507-30472-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115480851507-30472-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:11:41.742877 3381 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:11:41.743191 3381 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:11:41.743227 3381 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:11:41.745314 3381 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:11:41.745493 3381 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:11:41.745548 3381 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.194.1
I20260807 15:11:41.748200 3381 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.194.20:34113
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115480851507-30472-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115480851507-30472-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.1.keytab
--principal=kudu/127.29.194.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.194.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115480851507-30472-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.194.1
--webserver_port=0
--tserver_master_addrs=127.29.194.62:36595
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3381
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.194.1
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115480851507-30472-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:11:41.748486 3381 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:11:41.748709 3381 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:11:41.754424 3389 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:11:41.754427 3392 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:11:41.754429 3390 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:11:41.754853 3381 server_base.cc:1034] running on GCE node
I20260807 15:11:41.755080 3381 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:11:41.755339 3381 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:11:41.756577 3381 hybrid_clock.cc:648] HybridClock initialized: now 1786115501756541 us; error 29 us; skew 500 ppm
Aug 07 15:11:41 dist-test-slave-42z9 krb5kdc[3228](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115501, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.194.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:11:41.760546 3381 init.cc:377] Logged in from keytab as kudu/127.29.194.1@KRBTEST.COM (short username kudu)
I20260807 15:11:41.761232 3381 webserver.cc:460] Webserver started at http://127.29.194.1:38553/ using document root <none> and password file <none>
I20260807 15:11:41.761552 3381 fs_manager.cc:359] Metadata directory not provided
I20260807 15:11:41.761608 3381 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:11:41.761719 3381 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:11:41.762879 3381 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115480851507-30472-0/minicluster-data/ts-0/data/instance:
uuid: "546f2994b17947f1943351eacf54ca11"
format_stamp: "Formatted at 2026-08-07 15:11:41 on dist-test-slave-42z9"
server_key: "d1bc895eb82cc80e1384781d66c86d9d"
server_key_iv: "48c5643459f9ebbcc1003241dc9f0ab7"
server_key_version: "encryptionkey@0"
I20260807 15:11:41.763195 3381 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115480851507-30472-0/minicluster-data/ts-0/wal/instance:
uuid: "546f2994b17947f1943351eacf54ca11"
format_stamp: "Formatted at 2026-08-07 15:11:41 on dist-test-slave-42z9"
server_key: "d1bc895eb82cc80e1384781d66c86d9d"
server_key_iv: "48c5643459f9ebbcc1003241dc9f0ab7"
server_key_version: "encryptionkey@0"
I20260807 15:11:41.764604 3381 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260807 15:11:41.765630 3406 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:11:41.765817 3381 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 15:11:41.765913 3381 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115480851507-30472-0/minicluster-data/ts-0/data,/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115480851507-30472-0/minicluster-data/ts-0/wal
uuid: "546f2994b17947f1943351eacf54ca11"
format_stamp: "Formatted at 2026-08-07 15:11:41 on dist-test-slave-42z9"
server_key: "d1bc895eb82cc80e1384781d66c86d9d"
server_key_iv: "48c5643459f9ebbcc1003241dc9f0ab7"
server_key_version: "encryptionkey@0"
I20260807 15:11:41.765980 3381 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115480851507-30472-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115480851507-30472-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115480851507-30472-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:11:41.774492 3381 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:11:41.777683 3381 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:11:41.777835 3381 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:11:41.778110 3381 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:11:41.778524 3381 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:11:41.778566 3381 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:11:41.778590 3381 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:11:41.778601 3381 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:11:41.784408 3381 rpc_server.cc:307] RPC server started. Bound to: 127.29.194.1:40759
I20260807 15:11:41.784730 3556 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.194.1:40759 every 8 connection(s)
I20260807 15:11:41.784801 3381 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115480851507-30472-0/minicluster-data/ts-0/data/info.pb
Aug 07 15:11:41 dist-test-slave-42z9 krb5kdc[3228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115501, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.194.1@KRBTEST.COM for kudu/127.29.194.62@KRBTEST.COM
I20260807 15:11:41.793366 30472 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7fZSIL/build/release/bin/kudu as pid 3381
I20260807 15:11:41.793511 30472 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115480851507-30472-0/minicluster-data/ts-0/wal/instance
I20260807 15:11:41.793687 30472 external_mini_cluster.cc:1428] Setting key fb96a3749206e22439ae52374ce247b7
I20260807 15:11:41.795938 3343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:41.785699 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.194.1:38277 (local address 127.29.194.62:36595)
0807 15:11:41.785755 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:41.785758 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:41.786025 (+ 267us) server_negotiation.cc:407] Connection header received
0807 15:11:41.786970 (+ 945us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:41.786971 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:41.786984 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:41.787044 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:11:41.787704 (+ 660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:41.788248 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:41.789788 (+ 1540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:41.790021 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:41.790132 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:41.792612 (+ 2480us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:11:41.792614 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:11:41.792614 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:11:41.792622 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:11:41.794251 (+ 1629us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:41.794817 (+ 566us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:41.794819 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:41.794819 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:41.794912 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:41.795209 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:41.795209 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:41.795210 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:41.795327 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:11:41.795386 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0807 15:11:41.795774 (+ 388us) server_negotiation.cc:299] Negotiation successful
0807 15:11:41.795866 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260807 15:11:41.796161 3561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:41.785664 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:36595 (local address 127.29.194.1:38277)
0807 15:11:41.785836 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0807 15:11:41.785909 (+ 73us) client_negotiation.cc:174] Beginning negotiation
0807 15:11:41.786840 (+ 931us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:11:41.787111 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:11:41.787114 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:11:41.787396 (+ 282us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:11:41.787598 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:41.787603 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:41.788447 (+ 844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:41.788448 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:41.789683 (+ 1235us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:41.789690 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:41.790190 (+ 500us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:41.790191 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:41.790333 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:41.790940 (+ 607us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:11:41.790950 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:11:41.792483 (+ 1533us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:11:41.794351 (+ 1868us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:11:41.794362 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:11:41.794365 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:11:41.794721 (+ 356us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:11:41.794989 (+ 268us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:11:41.794990 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:11:41.794991 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:11:41.795144 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:11:41.795393 (+ 249us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:11:41.795400 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:11:41.795585 (+ 185us) client_negotiation.cc:769] Sending connection context
0807 15:11:41.795705 (+ 120us) client_negotiation.cc:240] Negotiation successful
0807 15:11:41.795901 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":98,"threads_started":1}
I20260807 15:11:41.796679 3557 heartbeater.cc:347] Connected to a master server at 127.29.194.62:36595
I20260807 15:11:41.796814 3557 heartbeater.cc:464] Registering TS with master...
I20260807 15:11:41.797032 3557 heartbeater.cc:511] Master 127.29.194.62:36595 requested a full tablet report, sending...
I20260807 15:11:41.797667 3291 ts_manager.cc:194] Registered new tserver with Master: 546f2994b17947f1943351eacf54ca11 (127.29.194.1:40759)
I20260807 15:11:41.798504 3291 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.194.1@KRBTEST.COM'} at 127.29.194.1:38277
WARNING: no policy specified for kudu/127.29.194.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.194.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.194.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.2.keytab.
Entry for principal kudu/127.29.194.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.2.keytab.
I20260807 15:11:41.826570 30472 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7fZSIL/build/release/bin/kudu
/tmp/dist-test-task7fZSIL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115480851507-30472-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115480851507-30472-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115480851507-30472-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115480851507-30472-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.194.2:0
--local_ip_for_outbound_sockets=127.29.194.2
--webserver_interface=127.29.194.2
--webserver_port=0
--tserver_master_addrs=127.29.194.62:36595
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.2.keytab
--principal=kudu/127.29.194.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.194.20:34113
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115480851507-30472-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115480851507-30472-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115480851507-30472-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:11:41.913300 3568 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:11:41.913585 3568 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:11:41.913626 3568 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:11:41.914888 3568 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:11:41.914933 3568 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:11:41.914973 3568 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.194.2
I20260807 15:11:41.916644 3568 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.194.20:34113
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115480851507-30472-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115480851507-30472-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.2.keytab
--principal=kudu/127.29.194.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.194.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115480851507-30472-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.194.2
--webserver_port=0
--tserver_master_addrs=127.29.194.62:36595
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3568
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.194.2
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115480851507-30472-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:11:41.916915 3568 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:11:41.917156 3568 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:11:41.922796 3580 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:11:41.922838 3568 server_base.cc:1034] running on GCE node
W20260807 15:11:41.922796 3576 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:11:41.922833 3575 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:11:41.923622 3568 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:11:41.923873 3568 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:11:41.925024 3568 hybrid_clock.cc:648] HybridClock initialized: now 1786115501924987 us; error 51 us; skew 500 ppm
Aug 07 15:11:41 dist-test-slave-42z9 krb5kdc[3228](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115501, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.194.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:11:41.927768 3568 init.cc:377] Logged in from keytab as kudu/127.29.194.2@KRBTEST.COM (short username kudu)
I20260807 15:11:41.928234 3568 webserver.cc:460] Webserver started at http://127.29.194.2:38037/ using document root <none> and password file <none>
I20260807 15:11:41.928452 3568 fs_manager.cc:359] Metadata directory not provided
I20260807 15:11:41.928494 3568 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:11:41.928588 3568 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:11:41.929697 3568 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115480851507-30472-0/minicluster-data/ts-1/data/instance:
uuid: "6b2b3fae1c874f8e9abd0586b100ffe6"
format_stamp: "Formatted at 2026-08-07 15:11:41 on dist-test-slave-42z9"
server_key: "275de0b925eb08a1de7f3990e3318920"
server_key_iv: "927a98fc4241e411d0e7b072a2dfdbcf"
server_key_version: "encryptionkey@0"
I20260807 15:11:41.930012 3568 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115480851507-30472-0/minicluster-data/ts-1/wal/instance:
uuid: "6b2b3fae1c874f8e9abd0586b100ffe6"
format_stamp: "Formatted at 2026-08-07 15:11:41 on dist-test-slave-42z9"
server_key: "275de0b925eb08a1de7f3990e3318920"
server_key_iv: "927a98fc4241e411d0e7b072a2dfdbcf"
server_key_version: "encryptionkey@0"
I20260807 15:11:41.931388 3568 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:11:41.932279 3593 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:11:41.932480 3568 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:11:41.932586 3568 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115480851507-30472-0/minicluster-data/ts-1/data,/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115480851507-30472-0/minicluster-data/ts-1/wal
uuid: "6b2b3fae1c874f8e9abd0586b100ffe6"
format_stamp: "Formatted at 2026-08-07 15:11:41 on dist-test-slave-42z9"
server_key: "275de0b925eb08a1de7f3990e3318920"
server_key_iv: "927a98fc4241e411d0e7b072a2dfdbcf"
server_key_version: "encryptionkey@0"
I20260807 15:11:41.932670 3568 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115480851507-30472-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115480851507-30472-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115480851507-30472-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:11:41.972321 3568 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:11:41.974747 3568 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:11:41.974876 3568 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:11:41.975158 3568 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:11:41.975497 3568 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:11:41.975538 3568 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:11:41.975562 3568 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:11:41.975586 3568 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:11:41.982179 3568 rpc_server.cc:307] RPC server started. Bound to: 127.29.194.2:36131
I20260807 15:11:41.982273 3740 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.194.2:36131 every 8 connection(s)
I20260807 15:11:41.982554 3568 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115480851507-30472-0/minicluster-data/ts-1/data/info.pb
I20260807 15:11:41.983938 30472 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7fZSIL/build/release/bin/kudu as pid 3568
I20260807 15:11:41.984078 30472 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115480851507-30472-0/minicluster-data/ts-1/wal/instance
I20260807 15:11:41.984380 30472 external_mini_cluster.cc:1428] Setting key 0d77ca930fc1228bf45513bac91ba30a
Aug 07 15:11:41 dist-test-slave-42z9 krb5kdc[3228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115501, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.194.2@KRBTEST.COM for kudu/127.29.194.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.29.194.3@KRBTEST.COM; defaulting to no policy
I20260807 15:11:41.993846 3343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:41.983492 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.194.2:36841 (local address 127.29.194.62:36595)
0807 15:11:41.983576 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:41.983579 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:41.983600 (+ 21us) server_negotiation.cc:407] Connection header received
0807 15:11:41.984639 (+ 1039us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:41.984641 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:41.984654 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:41.984727 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:11:41.985579 (+ 852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:41.986262 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:41.987724 (+ 1462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:41.987924 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:41.987984 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:41.990576 (+ 2592us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:11:41.990578 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:11:41.990578 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:11:41.990588 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:11:41.992196 (+ 1608us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:41.992700 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:41.992701 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:41.992701 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:41.992761 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:41.992999 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:41.993000 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:41.993000 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:41.993131 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:11:41.993188 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0807 15:11:41.993657 (+ 469us) server_negotiation.cc:299] Negotiation successful
0807 15:11:41.993748 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260807 15:11:41.994045 3745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:41.983310 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:36595 (local address 127.29.194.2:36841)
0807 15:11:41.983483 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0807 15:11:41.983528 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0807 15:11:41.984503 (+ 975us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:11:41.984821 (+ 318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:11:41.984825 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:11:41.985186 (+ 361us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:11:41.985459 (+ 273us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:41.985465 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:41.986406 (+ 941us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:41.986407 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:41.987623 (+ 1216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:41.987626 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:41.988001 (+ 375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:41.988002 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:41.988154 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:41.988679 (+ 525us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:11:41.988688 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:11:41.990435 (+ 1747us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:11:41.992304 (+ 1869us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:11:41.992316 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:11:41.992319 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:11:41.992621 (+ 302us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:11:41.992819 (+ 198us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:11:41.992820 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:11:41.992820 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:11:41.992942 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:11:41.993187 (+ 245us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:11:41.993194 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:11:41.993507 (+ 313us) client_negotiation.cc:769] Sending connection context
0807 15:11:41.993624 (+ 117us) client_negotiation.cc:240] Negotiation successful
0807 15:11:41.993809 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":76,"threads_started":1}
I20260807 15:11:41.994514 3741 heartbeater.cc:347] Connected to a master server at 127.29.194.62:36595
I20260807 15:11:41.994624 3741 heartbeater.cc:464] Registering TS with master...
I20260807 15:11:41.994813 3741 heartbeater.cc:511] Master 127.29.194.62:36595 requested a full tablet report, sending...
I20260807 15:11:41.995220 3291 ts_manager.cc:194] Registered new tserver with Master: 6b2b3fae1c874f8e9abd0586b100ffe6 (127.29.194.2:36131)
I20260807 15:11:41.995579 3291 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.194.2@KRBTEST.COM'} at 127.29.194.2:36841
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.194.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.194.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.3.keytab.
Entry for principal kudu/127.29.194.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.3.keytab.
I20260807 15:11:42.023448 30472 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7fZSIL/build/release/bin/kudu
/tmp/dist-test-task7fZSIL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115480851507-30472-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115480851507-30472-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115480851507-30472-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115480851507-30472-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.194.3:0
--local_ip_for_outbound_sockets=127.29.194.3
--webserver_interface=127.29.194.3
--webserver_port=0
--tserver_master_addrs=127.29.194.62:36595
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.3.keytab
--principal=kudu/127.29.194.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.194.20:34113
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115480851507-30472-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115480851507-30472-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115480851507-30472-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:11:42.122503 3750 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:11:42.122751 3750 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:11:42.122777 3750 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:11:42.124105 3750 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:11:42.124167 3750 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:11:42.124198 3750 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.194.3
I20260807 15:11:42.126384 3750 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.194.20:34113
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115480851507-30472-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115480851507-30472-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.3.keytab
--principal=kudu/127.29.194.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.194.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115480851507-30472-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.194.3
--webserver_port=0
--tserver_master_addrs=127.29.194.62:36595
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3750
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.194.3
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115480851507-30472-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:11:42.126732 3750 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:11:42.127050 3750 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:11:42.133168 3758 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:11:42.133162 3755 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:11:42.133162 3756 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:11:42.133388 3750 server_base.cc:1034] running on GCE node
I20260807 15:11:42.133699 3750 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:11:42.133968 3750 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:11:42.135174 3750 hybrid_clock.cc:648] HybridClock initialized: now 1786115502135143 us; error 49 us; skew 500 ppm
Aug 07 15:11:42 dist-test-slave-42z9 krb5kdc[3228](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115502, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.194.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:11:42.139076 3750 init.cc:377] Logged in from keytab as kudu/127.29.194.3@KRBTEST.COM (short username kudu)
I20260807 15:11:42.139755 3750 webserver.cc:460] Webserver started at http://127.29.194.3:39919/ using document root <none> and password file <none>
I20260807 15:11:42.140044 3750 fs_manager.cc:359] Metadata directory not provided
I20260807 15:11:42.140100 3750 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:11:42.140230 3750 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:11:42.141888 3750 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115480851507-30472-0/minicluster-data/ts-2/data/instance:
uuid: "e7e16408f8ef456883f1bee6317ba4b5"
format_stamp: "Formatted at 2026-08-07 15:11:42 on dist-test-slave-42z9"
server_key: "c84fef18f01e6acd73752ae1025f48d4"
server_key_iv: "fc497c6c63935573d170c933bc4407a0"
server_key_version: "encryptionkey@0"
I20260807 15:11:42.142422 3750 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115480851507-30472-0/minicluster-data/ts-2/wal/instance:
uuid: "e7e16408f8ef456883f1bee6317ba4b5"
format_stamp: "Formatted at 2026-08-07 15:11:42 on dist-test-slave-42z9"
server_key: "c84fef18f01e6acd73752ae1025f48d4"
server_key_iv: "fc497c6c63935573d170c933bc4407a0"
server_key_version: "encryptionkey@0"
I20260807 15:11:42.143978 3750 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 15:11:42.144963 3773 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:11:42.145383 3750 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:11:42.145594 3750 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115480851507-30472-0/minicluster-data/ts-2/data,/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115480851507-30472-0/minicluster-data/ts-2/wal
uuid: "e7e16408f8ef456883f1bee6317ba4b5"
format_stamp: "Formatted at 2026-08-07 15:11:42 on dist-test-slave-42z9"
server_key: "c84fef18f01e6acd73752ae1025f48d4"
server_key_iv: "fc497c6c63935573d170c933bc4407a0"
server_key_version: "encryptionkey@0"
I20260807 15:11:42.145767 3750 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115480851507-30472-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115480851507-30472-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115480851507-30472-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:11:42.174765 3750 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:11:42.177094 3750 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:11:42.177265 3750 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:11:42.177597 3750 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:11:42.177963 3750 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:11:42.177999 3750 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:11:42.178053 3750 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:11:42.178078 3750 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:11:42.185137 3750 rpc_server.cc:307] RPC server started. Bound to: 127.29.194.3:37773
I20260807 15:11:42.185242 3946 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.194.3:37773 every 8 connection(s)
I20260807 15:11:42.185642 3750 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115480851507-30472-0/minicluster-data/ts-2/data/info.pb
I20260807 15:11:42.191877 30472 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7fZSIL/build/release/bin/kudu as pid 3750
I20260807 15:11:42.191975 30472 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115480851507-30472-0/minicluster-data/ts-2/wal/instance
I20260807 15:11:42.192186 30472 external_mini_cluster.cc:1428] Setting key e265c532da3440e7595f00cb287562fe
Aug 07 15:11:42 dist-test-slave-42z9 krb5kdc[3228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115502, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.194.3@KRBTEST.COM for kudu/127.29.194.62@KRBTEST.COM
I20260807 15:11:42.196461 3343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:42.186488 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.194.3:42523 (local address 127.29.194.62:36595)
0807 15:11:42.186545 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:42.186547 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:42.186732 (+ 185us) server_negotiation.cc:407] Connection header received
0807 15:11:42.187784 (+ 1052us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:42.187786 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:42.187798 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:42.187870 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:11:42.188699 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:42.189243 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:42.190449 (+ 1206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:42.190659 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:42.190719 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:42.193363 (+ 2644us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:11:42.193364 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:11:42.193364 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:11:42.193373 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:11:42.194963 (+ 1590us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:42.195444 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:42.195445 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:42.195446 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:42.195510 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:42.195789 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:42.195790 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:42.195790 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:42.195921 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:11:42.195977 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0807 15:11:42.196273 (+ 296us) server_negotiation.cc:299] Negotiation successful
0807 15:11:42.196364 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260807 15:11:42.196677 3951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:42.186413 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:36595 (local address 127.29.194.3:42523)
0807 15:11:42.186604 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0807 15:11:42.186652 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0807 15:11:42.187621 (+ 969us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:11:42.188034 (+ 413us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:11:42.188038 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:11:42.188344 (+ 306us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:11:42.188561 (+ 217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:42.188566 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:42.189369 (+ 803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:42.189370 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:42.190335 (+ 965us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:42.190339 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:42.190772 (+ 433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:42.190773 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:42.190913 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:42.191492 (+ 579us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:11:42.191502 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:11:42.193259 (+ 1757us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:11:42.195089 (+ 1830us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:11:42.195100 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:11:42.195103 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:11:42.195366 (+ 263us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:11:42.195612 (+ 246us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:11:42.195612 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:11:42.195613 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:11:42.195725 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:11:42.195982 (+ 257us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:11:42.195989 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:11:42.196144 (+ 155us) client_negotiation.cc:769] Sending connection context
0807 15:11:42.196243 (+ 99us) client_negotiation.cc:240] Negotiation successful
0807 15:11:42.196424 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":92,"threads_started":1}
I20260807 15:11:42.197335 3947 heartbeater.cc:347] Connected to a master server at 127.29.194.62:36595
I20260807 15:11:42.197557 3947 heartbeater.cc:464] Registering TS with master...
I20260807 15:11:42.197822 3947 heartbeater.cc:511] Master 127.29.194.62:36595 requested a full tablet report, sending...
I20260807 15:11:42.198434 3291 ts_manager.cc:194] Registered new tserver with Master: e7e16408f8ef456883f1bee6317ba4b5 (127.29.194.3:37773)
I20260807 15:11:42.199119 3291 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.194.3@KRBTEST.COM'} at 127.29.194.3:42523
I20260807 15:11:42.205194 30472 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 07 15:11:42 dist-test-slave-42z9 krb5kdc[3228](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115502, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 07 15:11:42 dist-test-slave-42z9 krb5kdc[3228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115502, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.194.1@KRBTEST.COM
I20260807 15:11:42.233494 3957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:42.224288 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38406 (local address 127.29.194.1:40759)
0807 15:11:42.224568 (+ 280us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:42.224574 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:42.224584 (+ 10us) server_negotiation.cc:407] Connection header received
0807 15:11:42.224611 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:42.224614 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:42.224629 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:42.224702 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:11:42.225347 (+ 645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:42.226257 (+ 910us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:42.227087 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:42.227426 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:42.227485 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:42.230111 (+ 2626us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:11:42.230124 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:11:42.230125 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:11:42.230132 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:11:42.232218 (+ 2086us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:42.232612 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:42.232615 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:42.232615 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:42.232694 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:42.232872 (+ 178us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:42.232873 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:42.232873 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:42.233033 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:11:42.233177 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0807 15:11:42.233267 (+ 90us) server_negotiation.cc:299] Negotiation successful
0807 15:11:42.233358 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":223,"thread_start_us":104,"threads_started":1}
Aug 07 15:11:42 dist-test-slave-42z9 krb5kdc[3228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115502, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.194.62@KRBTEST.COM
I20260807 15:11:42.244956 3343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:42.235437 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41070 (local address 127.29.194.62:36595)
0807 15:11:42.235528 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:42.235531 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:42.235540 (+ 9us) server_negotiation.cc:407] Connection header received
0807 15:11:42.235571 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:42.235572 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:42.235584 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:42.235635 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:11:42.236374 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:42.236943 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:42.237708 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:42.237902 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:42.237959 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:42.241382 (+ 3423us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:11:42.241384 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:11:42.241385 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:11:42.241396 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:11:42.243552 (+ 2156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:42.244046 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:42.244047 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:42.244048 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:42.244114 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:42.244286 (+ 172us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:42.244286 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:42.244287 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:42.244503 (+ 216us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:11:42.244609 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0807 15:11:42.244734 (+ 125us) server_negotiation.cc:299] Negotiation successful
0807 15:11:42.244837 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
W20260807 15:11:42.245573 3291 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41070
I20260807 15:11:42.246308 30472 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7fZSIL/build/release/bin/kudu with pid 3381
I20260807 15:11:42.258407 30472 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7fZSIL/build/release/bin/kudu with pid 3568
I20260807 15:11:42.271963 30472 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7fZSIL/build/release/bin/kudu with pid 3750
I20260807 15:11:42.316210 30472 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7fZSIL/build/release/bin/kudu with pid 3246
2026-08-07T15:11:42Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (2266 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115480851507-30472-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 15:11:42 dist-test-slave-42z9 krb5kdc[3964](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 15:11:42 dist-test-slave-42z9 krb5kdc[3964](info): set up 2 sockets
Aug 07 15:11:42 dist-test-slave-42z9 krb5kdc[3964](info): commencing operation
krb5kdc: starting...
W20260807 15:11:43.496749 30472 mini_kdc.cc:121] Time spent starting KDC: real 1.148s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115480851507-30472-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115480851507-30472-0/krb5kdc/test-user.keytab.
Aug 07 15:11:43 dist-test-slave-42z9 krb5kdc[3964](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115503, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T15:11:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T15:11:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.194.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.194.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.194.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.62.keytab.
Entry for principal kudu/127.29.194.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.62.keytab.
I20260807 15:11:43.638465 30472 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7fZSIL/build/release/bin/kudu
/tmp/dist-test-task7fZSIL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115480851507-30472-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115480851507-30472-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115480851507-30472-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115480851507-30472-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.194.62:39805
--webserver_interface=127.29.194.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.62.keytab
--principal=kudu/127.29.194.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.194.20:46193
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.194.62:39805
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115480851507-30472-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115480851507-30472-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115480851507-30472-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:11:43.727622 3986 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:11:43.727867 3986 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:11:43.727893 3986 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:11:43.729135 3986 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:11:43.729184 3986 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:11:43.729198 3986 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:11:43.729210 3986 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:11:43.729220 3986 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:11:43.730994 3986 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.194.20:46193
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115480851507-30472-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115480851507-30472-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.194.62:39805
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.62.keytab
--principal=kudu/127.29.194.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.194.62:39805
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115480851507-30472-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.194.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3986
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115480851507-30472-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:11:43.731257 3986 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:11:43.731487 3986 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:11:43.736652 3999 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:11:43.736761 3995 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:11:43.736792 3997 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:11:43.736810 3986 server_base.cc:1034] running on GCE node
I20260807 15:11:43.737246 3986 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:11:43.737632 3986 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:11:43.738804 3986 hybrid_clock.cc:648] HybridClock initialized: now 1786115503738792 us; error 32 us; skew 500 ppm
Aug 07 15:11:43 dist-test-slave-42z9 krb5kdc[3964](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115503, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.194.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:11:43.745775 3986 init.cc:377] Logged in from keytab as kudu/127.29.194.62@KRBTEST.COM (short username kudu)
I20260807 15:11:43.746243 3986 fs_manager.cc:359] Metadata directory not provided
I20260807 15:11:43.746306 3986 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:11:43.746450 3986 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:11:43.747632 3986 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115480851507-30472-0/minicluster-data/master-0/data/instance:
uuid: "8b96e2433d2b4768916a8bd803d35282"
format_stamp: "Formatted at 2026-08-07 15:11:43 on dist-test-slave-42z9"
server_key: "6b421a0476d35150706a50b910bd1c62"
server_key_iv: "c8ea91e0a40e1e806445f2547c0290fd"
server_key_version: "encryptionkey@0"
I20260807 15:11:43.747994 3986 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115480851507-30472-0/minicluster-data/master-0/wal/instance:
uuid: "8b96e2433d2b4768916a8bd803d35282"
format_stamp: "Formatted at 2026-08-07 15:11:43 on dist-test-slave-42z9"
server_key: "6b421a0476d35150706a50b910bd1c62"
server_key_iv: "c8ea91e0a40e1e806445f2547c0290fd"
server_key_version: "encryptionkey@0"
I20260807 15:11:43.749428 3986 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 15:11:43.750275 4008 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:11:43.750465 3986 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 15:11:43.750566 3986 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115480851507-30472-0/minicluster-data/master-0/data,/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115480851507-30472-0/minicluster-data/master-0/wal
uuid: "8b96e2433d2b4768916a8bd803d35282"
format_stamp: "Formatted at 2026-08-07 15:11:43 on dist-test-slave-42z9"
server_key: "6b421a0476d35150706a50b910bd1c62"
server_key_iv: "c8ea91e0a40e1e806445f2547c0290fd"
server_key_version: "encryptionkey@0"
I20260807 15:11:43.750643 3986 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115480851507-30472-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115480851507-30472-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115480851507-30472-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:11:43.779042 3986 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:11:43.781399 3986 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:11:43.781587 3986 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:11:43.786273 3986 rpc_server.cc:307] RPC server started. Bound to: 127.29.194.62:39805
I20260807 15:11:43.786346 4089 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.194.62:39805 every 8 connection(s)
I20260807 15:11:43.786636 3986 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115480851507-30472-0/minicluster-data/master-0/data/info.pb
I20260807 15:11:43.787551 4090 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:11:43.790097 4090 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8b96e2433d2b4768916a8bd803d35282: Bootstrap starting.
I20260807 15:11:43.790783 4090 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8b96e2433d2b4768916a8bd803d35282: Neither blocks nor log segments found. Creating new log.
I20260807 15:11:43.791072 4090 log.cc:824] T 00000000000000000000000000000000 P 8b96e2433d2b4768916a8bd803d35282: Log is configured to *not* fsync() on all Append() calls
I20260807 15:11:43.792276 4090 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8b96e2433d2b4768916a8bd803d35282: No bootstrap required, opened a new log
I20260807 15:11:43.794137 4090 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8b96e2433d2b4768916a8bd803d35282 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b96e2433d2b4768916a8bd803d35282" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 39805 } }
I20260807 15:11:43.794355 4090 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8b96e2433d2b4768916a8bd803d35282 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:11:43.794394 4090 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8b96e2433d2b4768916a8bd803d35282 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8b96e2433d2b4768916a8bd803d35282, State: Initialized, Role: FOLLOWER
I20260807 15:11:43.794597 4090 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8b96e2433d2b4768916a8bd803d35282 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b96e2433d2b4768916a8bd803d35282" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 39805 } }
I20260807 15:11:43.794697 4090 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8b96e2433d2b4768916a8bd803d35282 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:11:43.794730 4090 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8b96e2433d2b4768916a8bd803d35282 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:11:43.794775 4090 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8b96e2433d2b4768916a8bd803d35282 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:11:43.796118 4090 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8b96e2433d2b4768916a8bd803d35282 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b96e2433d2b4768916a8bd803d35282" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 39805 } }
I20260807 15:11:43.796304 4090 leader_election.cc:302] T 00000000000000000000000000000000 P 8b96e2433d2b4768916a8bd803d35282 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8b96e2433d2b4768916a8bd803d35282; no voters:
I20260807 15:11:43.796533 4090 leader_election.cc:288] T 00000000000000000000000000000000 P 8b96e2433d2b4768916a8bd803d35282 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 15:11:43.796603 4095 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8b96e2433d2b4768916a8bd803d35282 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:11:43.796617 30472 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7fZSIL/build/release/bin/kudu as pid 3986
I20260807 15:11:43.796734 30472 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115480851507-30472-0/minicluster-data/master-0/wal/instance
I20260807 15:11:43.796801 4095 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8b96e2433d2b4768916a8bd803d35282 [term 1 LEADER]: Becoming Leader. State: Replica: 8b96e2433d2b4768916a8bd803d35282, State: Running, Role: LEADER
I20260807 15:11:43.796876 4090 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8b96e2433d2b4768916a8bd803d35282 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:11:43.796917 30472 external_mini_cluster.cc:1428] Setting key 4168302e5cf97b7a5a407a933a973648
I20260807 15:11:43.796941 4095 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8b96e2433d2b4768916a8bd803d35282 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b96e2433d2b4768916a8bd803d35282" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 39805 } }
I20260807 15:11:43.797341 4095 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8b96e2433d2b4768916a8bd803d35282 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8b96e2433d2b4768916a8bd803d35282" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b96e2433d2b4768916a8bd803d35282" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 39805 } } }
I20260807 15:11:43.797415 4095 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8b96e2433d2b4768916a8bd803d35282 [sys.catalog]: This master's current role is: LEADER
I20260807 15:11:43.797585 4095 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8b96e2433d2b4768916a8bd803d35282 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8b96e2433d2b4768916a8bd803d35282. Latest consensus state: current_term: 1 leader_uuid: "8b96e2433d2b4768916a8bd803d35282" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b96e2433d2b4768916a8bd803d35282" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 39805 } } }
I20260807 15:11:43.797634 4095 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8b96e2433d2b4768916a8bd803d35282 [sys.catalog]: This master's current role is: LEADER
W20260807 15:11:43.799618 4122 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 8b96e2433d2b4768916a8bd803d35282: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 15:11:43.799686 4122 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 15:11:43.799777 4109 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:11:43.799978 4109 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:11:43.803930 4109 catalog_manager.cc:1347] Generated new cluster ID: a25c8d7a7d4646fd84caa4c08413c218
I20260807 15:11:43.804015 4109 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 07 15:11:43 dist-test-slave-42z9 krb5kdc[3964](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115503, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.194.62@KRBTEST.COM
I20260807 15:11:43.820181 4112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:43.797601 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36796 (local address 127.29.194.62:39805)
0807 15:11:43.798348 (+ 747us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:43.798358 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:43.798392 (+ 34us) server_negotiation.cc:407] Connection header received
0807 15:11:43.798625 (+ 233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:43.798633 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:43.798695 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:43.798853 (+ 158us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:11:43.799878 (+ 1025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:43.801251 (+ 1373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:43.802410 (+ 1159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:43.803299 (+ 889us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:43.803417 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:43.813544 (+ 10127us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:11:43.813551 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:11:43.813552 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:11:43.813570 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:11:43.815773 (+ 2203us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:43.817841 (+ 2068us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:43.817849 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:43.817850 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:43.818073 (+ 223us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:43.818354 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:43.818355 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:43.818355 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:43.818960 (+ 605us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:11:43.819212 (+ 252us) server_negotiation.cc:1035] Waiting for connection context
0807 15:11:43.819649 (+ 437us) server_negotiation.cc:299] Negotiation successful
0807 15:11:43.819867 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":697,"thread_start_us":69,"threads_started":1}
I20260807 15:11:43.855242 4109 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:11:43.856051 4109 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:11:43.866696 4109 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8b96e2433d2b4768916a8bd803d35282: Generated new TSK 0
I20260807 15:11:43.866954 4109 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.29.194.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.194.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.194.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.1.keytab.
Entry for principal kudu/127.29.194.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.1.keytab.
I20260807 15:11:43.905272 30472 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7fZSIL/build/release/bin/kudu
/tmp/dist-test-task7fZSIL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115480851507-30472-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115480851507-30472-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115480851507-30472-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115480851507-30472-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.194.1:0
--local_ip_for_outbound_sockets=127.29.194.1
--webserver_interface=127.29.194.1
--webserver_port=0
--tserver_master_addrs=127.29.194.62:39805
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.1.keytab
--principal=kudu/127.29.194.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.194.20:46193
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115480851507-30472-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115480851507-30472-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115480851507-30472-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:11:43.995870 4129 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:11:43.996120 4129 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:11:43.996147 4129 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:11:43.997532 4129 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:11:43.997589 4129 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:11:43.997622 4129 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.194.1
I20260807 15:11:43.999490 4129 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.194.20:46193
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115480851507-30472-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115480851507-30472-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.1.keytab
--principal=kudu/127.29.194.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.194.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115480851507-30472-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.194.1
--webserver_port=0
--tserver_master_addrs=127.29.194.62:39805
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4129
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.194.1
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115480851507-30472-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:11:43.999765 4129 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:11:44.000352 4129 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:11:44.006683 4137 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:11:44.006693 4140 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:11:44.006693 4135 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:11:44.006908 4129 server_base.cc:1034] running on GCE node
I20260807 15:11:44.007156 4129 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:11:44.007396 4129 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:11:44.008580 4129 hybrid_clock.cc:648] HybridClock initialized: now 1786115504008565 us; error 30 us; skew 500 ppm
Aug 07 15:11:44 dist-test-slave-42z9 krb5kdc[3964](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115504, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.194.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:11:44.011576 4129 init.cc:377] Logged in from keytab as kudu/127.29.194.1@KRBTEST.COM (short username kudu)
I20260807 15:11:44.011976 4129 fs_manager.cc:359] Metadata directory not provided
I20260807 15:11:44.012040 4129 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:11:44.012143 4129 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:11:44.013240 4129 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115480851507-30472-0/minicluster-data/ts-0/data/instance:
uuid: "e27fb434f4ca40a79d97563be7d5b77a"
format_stamp: "Formatted at 2026-08-07 15:11:44 on dist-test-slave-42z9"
server_key: "cca0bd9f735fda0a604f0176576dc8de"
server_key_iv: "47e18eea8a52541dcc5d459dfa5ab9e0"
server_key_version: "encryptionkey@0"
I20260807 15:11:44.013612 4129 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115480851507-30472-0/minicluster-data/ts-0/wal/instance:
uuid: "e27fb434f4ca40a79d97563be7d5b77a"
format_stamp: "Formatted at 2026-08-07 15:11:44 on dist-test-slave-42z9"
server_key: "cca0bd9f735fda0a604f0176576dc8de"
server_key_iv: "47e18eea8a52541dcc5d459dfa5ab9e0"
server_key_version: "encryptionkey@0"
I20260807 15:11:44.014968 4129 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260807 15:11:44.016083 4147 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:11:44.016373 4129 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 15:11:44.016494 4129 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115480851507-30472-0/minicluster-data/ts-0/data,/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115480851507-30472-0/minicluster-data/ts-0/wal
uuid: "e27fb434f4ca40a79d97563be7d5b77a"
format_stamp: "Formatted at 2026-08-07 15:11:44 on dist-test-slave-42z9"
server_key: "cca0bd9f735fda0a604f0176576dc8de"
server_key_iv: "47e18eea8a52541dcc5d459dfa5ab9e0"
server_key_version: "encryptionkey@0"
I20260807 15:11:44.016606 4129 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115480851507-30472-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115480851507-30472-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115480851507-30472-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:11:44.026837 4129 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:11:44.029106 4129 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:11:44.029227 4129 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:11:44.029464 4129 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:11:44.029814 4129 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:11:44.029850 4129 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:11:44.029871 4129 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:11:44.029883 4129 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:11:44.035945 4129 rpc_server.cc:307] RPC server started. Bound to: 127.29.194.1:36903
I20260807 15:11:44.036006 4310 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.194.1:36903 every 8 connection(s)
I20260807 15:11:44.036571 4129 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115480851507-30472-0/minicluster-data/ts-0/data/info.pb
I20260807 15:11:44.041977 30472 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7fZSIL/build/release/bin/kudu as pid 4129
I20260807 15:11:44.042104 30472 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115480851507-30472-0/minicluster-data/ts-0/wal/instance
I20260807 15:11:44.042332 30472 external_mini_cluster.cc:1428] Setting key e68a97b55975f0204a652b5c7d47e2f4
Aug 07 15:11:44 dist-test-slave-42z9 krb5kdc[3964](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115504, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.194.1@KRBTEST.COM for kudu/127.29.194.62@KRBTEST.COM
I20260807 15:11:44.049742 4112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:44.037902 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.194.1:44329 (local address 127.29.194.62:39805)
0807 15:11:44.038094 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:44.038096 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:44.038434 (+ 338us) server_negotiation.cc:407] Connection header received
0807 15:11:44.039313 (+ 879us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:44.039315 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:44.039346 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:44.039404 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:11:44.040494 (+ 1090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:44.041025 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:44.042141 (+ 1116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:44.042346 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:44.042406 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:44.045958 (+ 3552us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:11:44.045961 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:11:44.045961 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:11:44.045971 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:11:44.048087 (+ 2116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:44.048697 (+ 610us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:44.048699 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:44.048699 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:44.048765 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:44.049013 (+ 248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:44.049014 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:44.049014 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:44.049143 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:11:44.049202 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0807 15:11:44.049539 (+ 337us) server_negotiation.cc:299] Negotiation successful
0807 15:11:44.049642 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":153}
I20260807 15:11:44.049954 4316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:44.037823 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:39805 (local address 127.29.194.1:44329)
0807 15:11:44.038136 (+ 313us) negotiation.cc:107] Waiting for socket to connect
0807 15:11:44.038252 (+ 116us) client_negotiation.cc:174] Beginning negotiation
0807 15:11:44.039145 (+ 893us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:11:44.039499 (+ 354us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:11:44.039503 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:11:44.039837 (+ 334us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:11:44.040378 (+ 541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:44.040384 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:44.041121 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:44.041122 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:44.042036 (+ 914us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:44.042040 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:44.042430 (+ 390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:44.042431 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:44.042577 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:44.043523 (+ 946us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:11:44.043532 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:11:44.045734 (+ 2202us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:11:44.048223 (+ 2489us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:11:44.048237 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:11:44.048240 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:11:44.048617 (+ 377us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:11:44.048825 (+ 208us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:11:44.048826 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:11:44.048826 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:11:44.048953 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:11:44.049205 (+ 252us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:11:44.049213 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:11:44.049366 (+ 153us) client_negotiation.cc:769] Sending connection context
0807 15:11:44.049515 (+ 149us) client_negotiation.cc:240] Negotiation successful
0807 15:11:44.049714 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":259,"thread_start_us":131,"threads_started":1}
I20260807 15:11:44.050575 4313 heartbeater.cc:347] Connected to a master server at 127.29.194.62:39805
I20260807 15:11:44.050688 4313 heartbeater.cc:464] Registering TS with master...
I20260807 15:11:44.050855 4313 heartbeater.cc:511] Master 127.29.194.62:39805 requested a full tablet report, sending...
I20260807 15:11:44.051441 4041 ts_manager.cc:194] Registered new tserver with Master: e27fb434f4ca40a79d97563be7d5b77a (127.29.194.1:36903)
I20260807 15:11:44.052362 4041 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.194.1@KRBTEST.COM'} at 127.29.194.1:44329
WARNING: no policy specified for kudu/127.29.194.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.194.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.194.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.2.keytab.
Entry for principal kudu/127.29.194.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.2.keytab.
I20260807 15:11:44.078217 30472 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7fZSIL/build/release/bin/kudu
/tmp/dist-test-task7fZSIL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115480851507-30472-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115480851507-30472-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115480851507-30472-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115480851507-30472-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.194.2:0
--local_ip_for_outbound_sockets=127.29.194.2
--webserver_interface=127.29.194.2
--webserver_port=0
--tserver_master_addrs=127.29.194.62:39805
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.2.keytab
--principal=kudu/127.29.194.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.194.20:46193
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115480851507-30472-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115480851507-30472-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115480851507-30472-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:11:44.174733 4320 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:11:44.175027 4320 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:11:44.175071 4320 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:11:44.176430 4320 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:11:44.176482 4320 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:11:44.176527 4320 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.194.2
I20260807 15:11:44.178501 4320 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.194.20:46193
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115480851507-30472-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115480851507-30472-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.2.keytab
--principal=kudu/127.29.194.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.194.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115480851507-30472-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.194.2
--webserver_port=0
--tserver_master_addrs=127.29.194.62:39805
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4320
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.194.2
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115480851507-30472-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:11:44.178834 4320 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:11:44.179076 4320 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:11:44.184470 4329 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:11:44.184470 4333 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:11:44.184619 4320 server_base.cc:1034] running on GCE node
W20260807 15:11:44.184767 4328 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:11:44.185002 4320 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:11:44.185261 4320 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:11:44.186439 4320 hybrid_clock.cc:648] HybridClock initialized: now 1786115504186422 us; error 28 us; skew 500 ppm
Aug 07 15:11:44 dist-test-slave-42z9 krb5kdc[3964](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115504, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.194.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:11:44.189106 4320 init.cc:377] Logged in from keytab as kudu/127.29.194.2@KRBTEST.COM (short username kudu)
I20260807 15:11:44.189458 4320 fs_manager.cc:359] Metadata directory not provided
I20260807 15:11:44.189558 4320 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:11:44.189685 4320 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:11:44.190745 4320 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115480851507-30472-0/minicluster-data/ts-1/data/instance:
uuid: "0c16d4b5aeb1494a9b8dac028113f72f"
format_stamp: "Formatted at 2026-08-07 15:11:44 on dist-test-slave-42z9"
server_key: "8e25859d5dc3fcaa75912d687d678b9d"
server_key_iv: "a59ea927f7bb1fec1b786857cfc1266f"
server_key_version: "encryptionkey@0"
I20260807 15:11:44.191085 4320 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115480851507-30472-0/minicluster-data/ts-1/wal/instance:
uuid: "0c16d4b5aeb1494a9b8dac028113f72f"
format_stamp: "Formatted at 2026-08-07 15:11:44 on dist-test-slave-42z9"
server_key: "8e25859d5dc3fcaa75912d687d678b9d"
server_key_iv: "a59ea927f7bb1fec1b786857cfc1266f"
server_key_version: "encryptionkey@0"
I20260807 15:11:44.192412 4320 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 15:11:44.193207 4345 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:11:44.193382 4320 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:11:44.193496 4320 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115480851507-30472-0/minicluster-data/ts-1/data,/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115480851507-30472-0/minicluster-data/ts-1/wal
uuid: "0c16d4b5aeb1494a9b8dac028113f72f"
format_stamp: "Formatted at 2026-08-07 15:11:44 on dist-test-slave-42z9"
server_key: "8e25859d5dc3fcaa75912d687d678b9d"
server_key_iv: "a59ea927f7bb1fec1b786857cfc1266f"
server_key_version: "encryptionkey@0"
I20260807 15:11:44.193588 4320 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115480851507-30472-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115480851507-30472-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115480851507-30472-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:11:44.200753 4320 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:11:44.202850 4320 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:11:44.203038 4320 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:11:44.203285 4320 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:11:44.203624 4320 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:11:44.203660 4320 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:11:44.203688 4320 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:11:44.203713 4320 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:11:44.210216 4320 rpc_server.cc:307] RPC server started. Bound to: 127.29.194.2:37305
I20260807 15:11:44.210270 4504 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.194.2:37305 every 8 connection(s)
I20260807 15:11:44.210532 4320 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115480851507-30472-0/minicluster-data/ts-1/data/info.pb
I20260807 15:11:44.215703 30472 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7fZSIL/build/release/bin/kudu as pid 4320
I20260807 15:11:44.215816 30472 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115480851507-30472-0/minicluster-data/ts-1/wal/instance
I20260807 15:11:44.215984 30472 external_mini_cluster.cc:1428] Setting key a40fafb777e9d6805fbb0742574da1b7
Aug 07 15:11:44 dist-test-slave-42z9 krb5kdc[3964](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115504, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.194.2@KRBTEST.COM for kudu/127.29.194.62@KRBTEST.COM
I20260807 15:11:44.220866 4112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:44.211296 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.194.2:60779 (local address 127.29.194.62:39805)
0807 15:11:44.211404 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:44.211406 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:44.211565 (+ 159us) server_negotiation.cc:407] Connection header received
0807 15:11:44.212041 (+ 476us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:44.212042 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:44.212054 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:44.212117 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:11:44.213144 (+ 1027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:44.213743 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:44.214804 (+ 1061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:44.215002 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:44.215064 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:44.217435 (+ 2371us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:11:44.217436 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:11:44.217437 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:11:44.217470 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:11:44.218887 (+ 1417us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:44.219674 (+ 787us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:44.219677 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:44.219678 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:44.219766 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:44.220030 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:44.220032 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:44.220032 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:44.220178 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:11:44.220247 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0807 15:11:44.220619 (+ 372us) server_negotiation.cc:299] Negotiation successful
0807 15:11:44.220754 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260807 15:11:44.221369 4508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:44.211263 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:39805 (local address 127.29.194.2:60779)
0807 15:11:44.211434 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0807 15:11:44.211483 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0807 15:11:44.211952 (+ 469us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:11:44.212180 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:11:44.212183 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:11:44.212462 (+ 279us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:11:44.213019 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:44.213024 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:44.213855 (+ 831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:44.213856 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:44.214705 (+ 849us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:44.214708 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:44.215105 (+ 397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:44.215106 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:44.215222 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:44.215791 (+ 569us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:11:44.215800 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:11:44.217341 (+ 1541us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:11:44.219061 (+ 1720us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:11:44.219078 (+ 17us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:11:44.219081 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:11:44.219511 (+ 430us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:11:44.219834 (+ 323us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:11:44.219835 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:11:44.219836 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:11:44.219965 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:11:44.220277 (+ 312us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:11:44.220286 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:11:44.220479 (+ 193us) client_negotiation.cc:769] Sending connection context
0807 15:11:44.220587 (+ 108us) client_negotiation.cc:240] Negotiation successful
0807 15:11:44.220818 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":88,"threads_started":1}
I20260807 15:11:44.222188 4505 heartbeater.cc:347] Connected to a master server at 127.29.194.62:39805
I20260807 15:11:44.222334 4505 heartbeater.cc:464] Registering TS with master...
I20260807 15:11:44.222543 4505 heartbeater.cc:511] Master 127.29.194.62:39805 requested a full tablet report, sending...
I20260807 15:11:44.222966 4041 ts_manager.cc:194] Registered new tserver with Master: 0c16d4b5aeb1494a9b8dac028113f72f (127.29.194.2:37305)
I20260807 15:11:44.223380 4041 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.194.2@KRBTEST.COM'} at 127.29.194.2:60779
WARNING: no policy specified for kudu/127.29.194.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.194.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.194.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.3.keytab.
Entry for principal kudu/127.29.194.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.3.keytab.
I20260807 15:11:44.249963 30472 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7fZSIL/build/release/bin/kudu
/tmp/dist-test-task7fZSIL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115480851507-30472-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115480851507-30472-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115480851507-30472-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115480851507-30472-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.194.3:0
--local_ip_for_outbound_sockets=127.29.194.3
--webserver_interface=127.29.194.3
--webserver_port=0
--tserver_master_addrs=127.29.194.62:39805
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.3.keytab
--principal=kudu/127.29.194.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.194.20:46193
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115480851507-30472-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115480851507-30472-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115480851507-30472-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:11:44.338455 4511 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:11:44.338712 4511 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:11:44.338749 4511 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:11:44.340060 4511 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:11:44.340121 4511 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:11:44.340163 4511 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.194.3
I20260807 15:11:44.341931 4511 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.194.20:46193
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115480851507-30472-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115480851507-30472-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.3.keytab
--principal=kudu/127.29.194.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.194.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115480851507-30472-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.194.3
--webserver_port=0
--tserver_master_addrs=127.29.194.62:39805
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4511
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.194.3
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115480851507-30472-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:11:44.342211 4511 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:11:44.342463 4511 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:11:44.347764 4524 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:11:44.347874 4511 server_base.cc:1034] running on GCE node
W20260807 15:11:44.347771 4521 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:11:44.347801 4522 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:11:44.348275 4511 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:11:44.348542 4511 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:11:44.349731 4511 hybrid_clock.cc:648] HybridClock initialized: now 1786115504349715 us; error 29 us; skew 500 ppm
Aug 07 15:11:44 dist-test-slave-42z9 krb5kdc[3964](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115504, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.194.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:11:44.352490 4511 init.cc:377] Logged in from keytab as kudu/127.29.194.3@KRBTEST.COM (short username kudu)
I20260807 15:11:44.352885 4511 fs_manager.cc:359] Metadata directory not provided
I20260807 15:11:44.352941 4511 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:11:44.353096 4511 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:11:44.354265 4511 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115480851507-30472-0/minicluster-data/ts-2/data/instance:
uuid: "c64ad608952248c5be9d951d5be7a51d"
format_stamp: "Formatted at 2026-08-07 15:11:44 on dist-test-slave-42z9"
server_key: "0deaad0f5341e1c83c113e4f72900f8a"
server_key_iv: "1d10552b382be9b73e098b86a7cce0ed"
server_key_version: "encryptionkey@0"
I20260807 15:11:44.354628 4511 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115480851507-30472-0/minicluster-data/ts-2/wal/instance:
uuid: "c64ad608952248c5be9d951d5be7a51d"
format_stamp: "Formatted at 2026-08-07 15:11:44 on dist-test-slave-42z9"
server_key: "0deaad0f5341e1c83c113e4f72900f8a"
server_key_iv: "1d10552b382be9b73e098b86a7cce0ed"
server_key_version: "encryptionkey@0"
I20260807 15:11:44.355981 4511 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260807 15:11:44.356871 4532 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:11:44.357142 4511 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 15:11:44.357237 4511 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115480851507-30472-0/minicluster-data/ts-2/data,/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115480851507-30472-0/minicluster-data/ts-2/wal
uuid: "c64ad608952248c5be9d951d5be7a51d"
format_stamp: "Formatted at 2026-08-07 15:11:44 on dist-test-slave-42z9"
server_key: "0deaad0f5341e1c83c113e4f72900f8a"
server_key_iv: "1d10552b382be9b73e098b86a7cce0ed"
server_key_version: "encryptionkey@0"
I20260807 15:11:44.357306 4511 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115480851507-30472-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115480851507-30472-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115480851507-30472-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:11:44.381284 4511 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:11:44.383585 4511 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:11:44.383737 4511 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:11:44.383996 4511 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:11:44.384346 4511 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:11:44.384384 4511 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:11:44.384415 4511 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:11:44.384433 4511 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:11:44.390586 4511 rpc_server.cc:307] RPC server started. Bound to: 127.29.194.3:39501
I20260807 15:11:44.390673 4695 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.194.3:39501 every 8 connection(s)
I20260807 15:11:44.390928 4511 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115480851507-30472-0/minicluster-data/ts-2/data/info.pb
I20260807 15:11:44.396728 30472 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7fZSIL/build/release/bin/kudu as pid 4511
I20260807 15:11:44.396883 30472 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115480851507-30472-0/minicluster-data/ts-2/wal/instance
I20260807 15:11:44.397130 30472 external_mini_cluster.cc:1428] Setting key 27c08725796bcbe2163b146558ba25a0
Aug 07 15:11:44 dist-test-slave-42z9 krb5kdc[3964](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115504, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.194.3@KRBTEST.COM for kudu/127.29.194.62@KRBTEST.COM
I20260807 15:11:44.403136 4112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:44.391943 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.194.3:57813 (local address 127.29.194.62:39805)
0807 15:11:44.392058 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:44.392061 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:44.392118 (+ 57us) server_negotiation.cc:407] Connection header received
0807 15:11:44.392657 (+ 539us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:44.392659 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:44.392672 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:44.392749 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:11:44.393846 (+ 1097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:44.394377 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:44.395538 (+ 1161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:44.395886 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:44.395963 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:44.399320 (+ 3357us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:11:44.399321 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:11:44.399322 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:11:44.399345 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:11:44.401016 (+ 1671us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:44.401644 (+ 628us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:44.401645 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:44.401645 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:44.401711 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:44.402082 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:44.402084 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:44.402084 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:44.402308 (+ 224us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:11:44.402470 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0807 15:11:44.402951 (+ 481us) server_negotiation.cc:299] Negotiation successful
0807 15:11:44.403059 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260807 15:11:44.403343 4698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:44.391810 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:39805 (local address 127.29.194.3:57813)
0807 15:11:44.391986 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0807 15:11:44.392032 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0807 15:11:44.392520 (+ 488us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:11:44.392817 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:11:44.392821 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:11:44.393140 (+ 319us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:11:44.393731 (+ 591us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:44.393738 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:44.394514 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:44.394515 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:44.395365 (+ 850us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:44.395368 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:44.396064 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:44.396066 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:44.396288 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:44.397082 (+ 794us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:11:44.397098 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:11:44.399166 (+ 2068us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:11:44.401125 (+ 1959us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:11:44.401137 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:11:44.401139 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:11:44.401545 (+ 406us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:11:44.401805 (+ 260us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:11:44.401806 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:11:44.401807 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:11:44.401942 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:11:44.402449 (+ 507us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:11:44.402459 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:11:44.402751 (+ 292us) client_negotiation.cc:769] Sending connection context
0807 15:11:44.402873 (+ 122us) client_negotiation.cc:240] Negotiation successful
0807 15:11:44.403096 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":79,"threads_started":1}
I20260807 15:11:44.404043 4696 heartbeater.cc:347] Connected to a master server at 127.29.194.62:39805
I20260807 15:11:44.404155 4696 heartbeater.cc:464] Registering TS with master...
I20260807 15:11:44.404350 4696 heartbeater.cc:511] Master 127.29.194.62:39805 requested a full tablet report, sending...
I20260807 15:11:44.404865 4041 ts_manager.cc:194] Registered new tserver with Master: c64ad608952248c5be9d951d5be7a51d (127.29.194.3:39501)
I20260807 15:11:44.405263 4041 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.194.3@KRBTEST.COM'} at 127.29.194.3:57813
I20260807 15:11:44.410270 30472 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 15:11:44.417977 4112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:44.411848 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36808 (local address 127.29.194.62:39805)
0807 15:11:44.411947 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:44.411949 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:44.412120 (+ 171us) server_negotiation.cc:407] Connection header received
0807 15:11:44.412197 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:44.412198 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:44.412209 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:44.412258 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:11:44.412859 (+ 601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:44.413352 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:44.414086 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:44.414271 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:44.414329 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:44.415536 (+ 1207us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:11:44.415538 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:11:44.415538 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:11:44.415546 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:11:44.416800 (+ 1254us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:44.417136 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:44.417137 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:44.417137 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:44.417190 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:44.417359 (+ 169us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:44.417360 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:44.417361 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:44.417576 (+ 215us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:11:44.417671 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0807 15:11:44.417797 (+ 126us) server_negotiation.cc:299] Negotiation successful
0807 15:11:44.417881 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260807 15:11:44.420442 4041 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36808:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 15:11:44.423199 4041 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 15:11:44.431316 4710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:44.427029 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.1:36903 (local address 127.0.0.1:45464)
0807 15:11:44.427233 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0807 15:11:44.427275 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0807 15:11:44.427352 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:11:44.427801 (+ 449us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:11:44.427805 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:11:44.427807 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:11:44.427928 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:44.427930 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:44.429292 (+ 1362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:44.429293 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:44.430224 (+ 931us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:44.430226 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:44.431102 (+ 876us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:44.431103 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:44.431158 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:44.431161 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 15:11:44.431203 (+ 42us) client_negotiation.cc:240] Negotiation successful
0807 15:11:44.431232 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":175,"mutex_wait_us":55,"thread_start_us":85,"threads_started":1}
I20260807 15:11:44.431365 4711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:44.427172 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45464 (local address 127.29.194.1:36903)
0807 15:11:44.427354 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:44.427360 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:44.427371 (+ 11us) server_negotiation.cc:407] Connection header received
0807 15:11:44.427601 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:44.427603 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:44.427619 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:44.427699 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:11:44.428018 (+ 319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:44.429175 (+ 1157us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:44.430296 (+ 1121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:44.431016 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:44.431101 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:44.431159 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0807 15:11:44.431233 (+ 74us) server_negotiation.cc:299] Negotiation successful
0807 15:11:44.431276 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":152,"thread_start_us":79,"threads_started":1}
I20260807 15:11:44.431481 4712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:44.427216 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55510 (local address 127.29.194.3:39501)
0807 15:11:44.427448 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:44.427452 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:44.427672 (+ 220us) server_negotiation.cc:407] Connection header received
0807 15:11:44.427719 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:44.427721 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:44.427733 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:44.427801 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:11:44.428032 (+ 231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:44.428802 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:44.430313 (+ 1511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:44.430880 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:44.430948 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:44.431012 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0807 15:11:44.431358 (+ 346us) server_negotiation.cc:299] Negotiation successful
0807 15:11:44.431400 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":198,"thread_start_us":96,"threads_started":1}
I20260807 15:11:44.431317 4715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:44.427033 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.3:39501 (local address 127.0.0.1:55510)
0807 15:11:44.427599 (+ 566us) negotiation.cc:107] Waiting for socket to connect
0807 15:11:44.427607 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0807 15:11:44.427650 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:11:44.427857 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:11:44.427858 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:11:44.427861 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:11:44.427941 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:44.427943 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:44.428900 (+ 957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:44.428901 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:44.430219 (+ 1318us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:44.430222 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:44.430971 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:44.430972 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:44.431067 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:44.431085 (+ 18us) client_negotiation.cc:769] Sending connection context
0807 15:11:44.431154 (+ 69us) client_negotiation.cc:240] Negotiation successful
0807 15:11:44.431215 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":550,"mutex_wait_us":22,"thread_start_us":379,"threads_started":1}
I20260807 15:11:44.431609 4714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:44.427033 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.2:37305 (local address 127.0.0.1:36790)
0807 15:11:44.427491 (+ 458us) negotiation.cc:107] Waiting for socket to connect
0807 15:11:44.427501 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 15:11:44.427538 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:11:44.427801 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:11:44.427805 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:11:44.427809 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:11:44.427968 (+ 159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:44.427971 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:44.429299 (+ 1328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:44.429300 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:44.430391 (+ 1091us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:44.430394 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:44.431325 (+ 931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:44.431326 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:44.431387 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:44.431392 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 15:11:44.431439 (+ 47us) client_negotiation.cc:240] Negotiation successful
0807 15:11:44.431479 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":438,"mutex_wait_us":100,"thread_start_us":409,"threads_started":1}
I20260807 15:11:44.432255 4713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:44.427247 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36790 (local address 127.29.194.2:37305)
0807 15:11:44.427519 (+ 272us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:44.427522 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:44.427559 (+ 37us) server_negotiation.cc:407] Connection header received
0807 15:11:44.427600 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:44.427603 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:44.427619 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:44.427699 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:11:44.428047 (+ 348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:44.429174 (+ 1127us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:44.430473 (+ 1299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:44.431179 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:44.431242 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:44.431291 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0807 15:11:44.432111 (+ 820us) server_negotiation.cc:299] Negotiation successful
0807 15:11:44.432172 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":236,"thread_start_us":71,"threads_started":1}
I20260807 15:11:44.432345 4596 tablet_service.cc:1467] Processing CreateTablet for tablet 4f50f22492c6484ca89479aa0bcf9d40 (DEFAULT_TABLE table=test-table [id=231cd01cfaea43f996182d43bff33058]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:11:44.432345 4218 tablet_service.cc:1467] Processing CreateTablet for tablet 4f50f22492c6484ca89479aa0bcf9d40 (DEFAULT_TABLE table=test-table [id=231cd01cfaea43f996182d43bff33058]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:11:44.432756 4596 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4f50f22492c6484ca89479aa0bcf9d40. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:11:44.432756 4218 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4f50f22492c6484ca89479aa0bcf9d40. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:11:44.432940 4410 tablet_service.cc:1467] Processing CreateTablet for tablet 4f50f22492c6484ca89479aa0bcf9d40 (DEFAULT_TABLE table=test-table [id=231cd01cfaea43f996182d43bff33058]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:11:44.433202 4410 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4f50f22492c6484ca89479aa0bcf9d40. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:11:44.436630 4720 tablet_bootstrap.cc:492] T 4f50f22492c6484ca89479aa0bcf9d40 P c64ad608952248c5be9d951d5be7a51d: Bootstrap starting.
I20260807 15:11:44.437361 4717 tablet_bootstrap.cc:492] T 4f50f22492c6484ca89479aa0bcf9d40 P 0c16d4b5aeb1494a9b8dac028113f72f: Bootstrap starting.
I20260807 15:11:44.438112 4720 tablet_bootstrap.cc:654] T 4f50f22492c6484ca89479aa0bcf9d40 P c64ad608952248c5be9d951d5be7a51d: Neither blocks nor log segments found. Creating new log.
I20260807 15:11:44.438264 4717 tablet_bootstrap.cc:654] T 4f50f22492c6484ca89479aa0bcf9d40 P 0c16d4b5aeb1494a9b8dac028113f72f: Neither blocks nor log segments found. Creating new log.
I20260807 15:11:44.438807 4720 log.cc:824] T 4f50f22492c6484ca89479aa0bcf9d40 P c64ad608952248c5be9d951d5be7a51d: Log is configured to *not* fsync() on all Append() calls
I20260807 15:11:44.438863 4717 log.cc:824] T 4f50f22492c6484ca89479aa0bcf9d40 P 0c16d4b5aeb1494a9b8dac028113f72f: Log is configured to *not* fsync() on all Append() calls
I20260807 15:11:44.439288 4719 tablet_bootstrap.cc:492] T 4f50f22492c6484ca89479aa0bcf9d40 P e27fb434f4ca40a79d97563be7d5b77a: Bootstrap starting.
I20260807 15:11:44.439958 4717 tablet_bootstrap.cc:492] T 4f50f22492c6484ca89479aa0bcf9d40 P 0c16d4b5aeb1494a9b8dac028113f72f: No bootstrap required, opened a new log
I20260807 15:11:44.440042 4717 ts_tablet_manager.cc:1397] T 4f50f22492c6484ca89479aa0bcf9d40 P 0c16d4b5aeb1494a9b8dac028113f72f: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260807 15:11:44.440083 4720 tablet_bootstrap.cc:492] T 4f50f22492c6484ca89479aa0bcf9d40 P c64ad608952248c5be9d951d5be7a51d: No bootstrap required, opened a new log
I20260807 15:11:44.440083 4719 tablet_bootstrap.cc:654] T 4f50f22492c6484ca89479aa0bcf9d40 P e27fb434f4ca40a79d97563be7d5b77a: Neither blocks nor log segments found. Creating new log.
I20260807 15:11:44.440172 4720 ts_tablet_manager.cc:1397] T 4f50f22492c6484ca89479aa0bcf9d40 P c64ad608952248c5be9d951d5be7a51d: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.003s
I20260807 15:11:44.440737 4719 log.cc:824] T 4f50f22492c6484ca89479aa0bcf9d40 P e27fb434f4ca40a79d97563be7d5b77a: Log is configured to *not* fsync() on all Append() calls
I20260807 15:11:44.441828 4719 tablet_bootstrap.cc:492] T 4f50f22492c6484ca89479aa0bcf9d40 P e27fb434f4ca40a79d97563be7d5b77a: No bootstrap required, opened a new log
I20260807 15:11:44.441923 4719 ts_tablet_manager.cc:1397] T 4f50f22492c6484ca89479aa0bcf9d40 P e27fb434f4ca40a79d97563be7d5b77a: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260807 15:11:44.442150 4717 raft_consensus.cc:348] T 4f50f22492c6484ca89479aa0bcf9d40 P 0c16d4b5aeb1494a9b8dac028113f72f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e27fb434f4ca40a79d97563be7d5b77a" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 36903 } } peers { permanent_uuid: "c64ad608952248c5be9d951d5be7a51d" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 39501 } } peers { permanent_uuid: "0c16d4b5aeb1494a9b8dac028113f72f" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 37305 } }
I20260807 15:11:44.442339 4717 raft_consensus.cc:374] T 4f50f22492c6484ca89479aa0bcf9d40 P 0c16d4b5aeb1494a9b8dac028113f72f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:11:44.442283 4720 raft_consensus.cc:348] T 4f50f22492c6484ca89479aa0bcf9d40 P c64ad608952248c5be9d951d5be7a51d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e27fb434f4ca40a79d97563be7d5b77a" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 36903 } } peers { permanent_uuid: "c64ad608952248c5be9d951d5be7a51d" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 39501 } } peers { permanent_uuid: "0c16d4b5aeb1494a9b8dac028113f72f" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 37305 } }
I20260807 15:11:44.442385 4717 raft_consensus.cc:729] T 4f50f22492c6484ca89479aa0bcf9d40 P 0c16d4b5aeb1494a9b8dac028113f72f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0c16d4b5aeb1494a9b8dac028113f72f, State: Initialized, Role: FOLLOWER
I20260807 15:11:44.442471 4720 raft_consensus.cc:374] T 4f50f22492c6484ca89479aa0bcf9d40 P c64ad608952248c5be9d951d5be7a51d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:11:44.442509 4720 raft_consensus.cc:729] T 4f50f22492c6484ca89479aa0bcf9d40 P c64ad608952248c5be9d951d5be7a51d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c64ad608952248c5be9d951d5be7a51d, State: Initialized, Role: FOLLOWER
I20260807 15:11:44.442675 4720 consensus_queue.cc:260] T 4f50f22492c6484ca89479aa0bcf9d40 P c64ad608952248c5be9d951d5be7a51d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e27fb434f4ca40a79d97563be7d5b77a" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 36903 } } peers { permanent_uuid: "c64ad608952248c5be9d951d5be7a51d" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 39501 } } peers { permanent_uuid: "0c16d4b5aeb1494a9b8dac028113f72f" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 37305 } }
I20260807 15:11:44.442664 4717 consensus_queue.cc:260] T 4f50f22492c6484ca89479aa0bcf9d40 P 0c16d4b5aeb1494a9b8dac028113f72f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e27fb434f4ca40a79d97563be7d5b77a" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 36903 } } peers { permanent_uuid: "c64ad608952248c5be9d951d5be7a51d" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 39501 } } peers { permanent_uuid: "0c16d4b5aeb1494a9b8dac028113f72f" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 37305 } }
I20260807 15:11:44.443033 4720 ts_tablet_manager.cc:1428] T 4f50f22492c6484ca89479aa0bcf9d40 P c64ad608952248c5be9d951d5be7a51d: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260807 15:11:44.443112 4717 ts_tablet_manager.cc:1428] T 4f50f22492c6484ca89479aa0bcf9d40 P 0c16d4b5aeb1494a9b8dac028113f72f: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260807 15:11:44.443172 4505 heartbeater.cc:503] Master 127.29.194.62:39805 was elected leader, sending a full tablet report...
I20260807 15:11:44.443265 4696 heartbeater.cc:503] Master 127.29.194.62:39805 was elected leader, sending a full tablet report...
I20260807 15:11:44.444485 4719 raft_consensus.cc:348] T 4f50f22492c6484ca89479aa0bcf9d40 P e27fb434f4ca40a79d97563be7d5b77a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e27fb434f4ca40a79d97563be7d5b77a" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 36903 } } peers { permanent_uuid: "c64ad608952248c5be9d951d5be7a51d" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 39501 } } peers { permanent_uuid: "0c16d4b5aeb1494a9b8dac028113f72f" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 37305 } }
I20260807 15:11:44.444648 4719 raft_consensus.cc:374] T 4f50f22492c6484ca89479aa0bcf9d40 P e27fb434f4ca40a79d97563be7d5b77a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:11:44.444680 4719 raft_consensus.cc:729] T 4f50f22492c6484ca89479aa0bcf9d40 P e27fb434f4ca40a79d97563be7d5b77a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e27fb434f4ca40a79d97563be7d5b77a, State: Initialized, Role: FOLLOWER
I20260807 15:11:44.444974 4719 consensus_queue.cc:260] T 4f50f22492c6484ca89479aa0bcf9d40 P e27fb434f4ca40a79d97563be7d5b77a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e27fb434f4ca40a79d97563be7d5b77a" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 36903 } } peers { permanent_uuid: "c64ad608952248c5be9d951d5be7a51d" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 39501 } } peers { permanent_uuid: "0c16d4b5aeb1494a9b8dac028113f72f" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 37305 } }
I20260807 15:11:44.445541 4719 ts_tablet_manager.cc:1428] T 4f50f22492c6484ca89479aa0bcf9d40 P e27fb434f4ca40a79d97563be7d5b77a: Time spent starting tablet: real 0.004s user 0.000s sys 0.003s
I20260807 15:11:44.445560 4313 heartbeater.cc:503] Master 127.29.194.62:39805 was elected leader, sending a full tablet report...
W20260807 15:11:44.461227 4507 tablet.cc:2369] T 4f50f22492c6484ca89479aa0bcf9d40 P 0c16d4b5aeb1494a9b8dac028113f72f: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 15:11:44.537734 4314 tablet.cc:2369] T 4f50f22492c6484ca89479aa0bcf9d40 P e27fb434f4ca40a79d97563be7d5b77a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 15:11:44.564221 4729 raft_consensus.cc:482] T 4f50f22492c6484ca89479aa0bcf9d40 P 0c16d4b5aeb1494a9b8dac028113f72f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 15:11:44.564366 4729 raft_consensus.cc:504] T 4f50f22492c6484ca89479aa0bcf9d40 P 0c16d4b5aeb1494a9b8dac028113f72f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e27fb434f4ca40a79d97563be7d5b77a" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 36903 } } peers { permanent_uuid: "c64ad608952248c5be9d951d5be7a51d" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 39501 } } peers { permanent_uuid: "0c16d4b5aeb1494a9b8dac028113f72f" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 37305 } }
I20260807 15:11:44.564785 4729 leader_election.cc:288] T 4f50f22492c6484ca89479aa0bcf9d40 P 0c16d4b5aeb1494a9b8dac028113f72f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e27fb434f4ca40a79d97563be7d5b77a (127.29.194.1:36903), c64ad608952248c5be9d951d5be7a51d (127.29.194.3:39501)
I20260807 15:11:44.567813 4712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:44.565241 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.194.2:35931 (local address 127.29.194.3:39501)
0807 15:11:44.565287 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:44.565289 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:44.565300 (+ 11us) server_negotiation.cc:407] Connection header received
0807 15:11:44.565333 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:44.565335 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:44.565344 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:44.565433 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:11:44.565634 (+ 201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:44.566132 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:44.567090 (+ 958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:44.567487 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:44.567552 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:44.567562 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0807 15:11:44.567703 (+ 141us) server_negotiation.cc:299] Negotiation successful
0807 15:11:44.567738 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260807 15:11:44.567813 4508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:44.565046 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.3:39501 (local address 127.29.194.2:35931)
0807 15:11:44.565132 (+ 86us) negotiation.cc:107] Waiting for socket to connect
0807 15:11:44.565149 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 15:11:44.565219 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:11:44.565492 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:11:44.565494 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:11:44.565497 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:11:44.565569 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:44.565571 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:44.566224 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:44.566225 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:44.567004 (+ 779us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:44.567006 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:44.567569 (+ 563us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:44.567570 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:44.567643 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:44.567646 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 15:11:44.567692 (+ 46us) client_negotiation.cc:240] Negotiation successful
0807 15:11:44.567735 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":48}
I20260807 15:11:44.567999 4711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:44.565173 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.194.2:58845 (local address 127.29.194.1:36903)
0807 15:11:44.565229 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:44.565231 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:44.565636 (+ 405us) server_negotiation.cc:407] Connection header received
0807 15:11:44.565694 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:44.565695 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:44.565705 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:44.565758 (+ 53us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:11:44.565928 (+ 170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:44.566406 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:44.567312 (+ 906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:44.567674 (+ 362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:44.567735 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:44.567742 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0807 15:11:44.567879 (+ 137us) server_negotiation.cc:299] Negotiation successful
0807 15:11:44.567918 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260807 15:11:44.568194 4732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:44.565151 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.1:36903 (local address 127.29.194.2:58845)
0807 15:11:44.565543 (+ 392us) negotiation.cc:107] Waiting for socket to connect
0807 15:11:44.565556 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 15:11:44.565640 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:11:44.565773 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:11:44.565774 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:11:44.565776 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:11:44.565845 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:44.565846 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:44.566487 (+ 641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:44.566488 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:44.567230 (+ 742us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:44.567232 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:44.567751 (+ 519us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:44.567752 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:44.567820 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:44.567823 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 15:11:44.568023 (+ 200us) client_negotiation.cc:240] Negotiation successful
0807 15:11:44.568117 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":365,"thread_start_us":266,"threads_started":1}
I20260807 15:11:44.568220 4626 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4f50f22492c6484ca89479aa0bcf9d40" candidate_uuid: "0c16d4b5aeb1494a9b8dac028113f72f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c64ad608952248c5be9d951d5be7a51d" is_pre_election: true
I20260807 15:11:44.568399 4626 raft_consensus.cc:2454] T 4f50f22492c6484ca89479aa0bcf9d40 P c64ad608952248c5be9d951d5be7a51d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0c16d4b5aeb1494a9b8dac028113f72f in term 0.
I20260807 15:11:44.568526 4250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4f50f22492c6484ca89479aa0bcf9d40" candidate_uuid: "0c16d4b5aeb1494a9b8dac028113f72f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e27fb434f4ca40a79d97563be7d5b77a" is_pre_election: true
I20260807 15:11:44.568691 4250 raft_consensus.cc:2454] T 4f50f22492c6484ca89479aa0bcf9d40 P e27fb434f4ca40a79d97563be7d5b77a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0c16d4b5aeb1494a9b8dac028113f72f in term 0.
I20260807 15:11:44.568677 4352 leader_election.cc:302] T 4f50f22492c6484ca89479aa0bcf9d40 P 0c16d4b5aeb1494a9b8dac028113f72f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0c16d4b5aeb1494a9b8dac028113f72f, c64ad608952248c5be9d951d5be7a51d; no voters:
I20260807 15:11:44.568823 4729 raft_consensus.cc:2781] T 4f50f22492c6484ca89479aa0bcf9d40 P 0c16d4b5aeb1494a9b8dac028113f72f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 15:11:44.568905 4729 raft_consensus.cc:482] T 4f50f22492c6484ca89479aa0bcf9d40 P 0c16d4b5aeb1494a9b8dac028113f72f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 15:11:44.568931 4729 raft_consensus.cc:3037] T 4f50f22492c6484ca89479aa0bcf9d40 P 0c16d4b5aeb1494a9b8dac028113f72f [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:11:44.569758 4729 raft_consensus.cc:504] T 4f50f22492c6484ca89479aa0bcf9d40 P 0c16d4b5aeb1494a9b8dac028113f72f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e27fb434f4ca40a79d97563be7d5b77a" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 36903 } } peers { permanent_uuid: "c64ad608952248c5be9d951d5be7a51d" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 39501 } } peers { permanent_uuid: "0c16d4b5aeb1494a9b8dac028113f72f" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 37305 } }
I20260807 15:11:44.569908 4729 leader_election.cc:288] T 4f50f22492c6484ca89479aa0bcf9d40 P 0c16d4b5aeb1494a9b8dac028113f72f [CANDIDATE]: Term 1 election: Requested vote from peers e27fb434f4ca40a79d97563be7d5b77a (127.29.194.1:36903), c64ad608952248c5be9d951d5be7a51d (127.29.194.3:39501)
I20260807 15:11:44.570075 4626 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4f50f22492c6484ca89479aa0bcf9d40" candidate_uuid: "0c16d4b5aeb1494a9b8dac028113f72f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c64ad608952248c5be9d951d5be7a51d"
I20260807 15:11:44.570087 4250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4f50f22492c6484ca89479aa0bcf9d40" candidate_uuid: "0c16d4b5aeb1494a9b8dac028113f72f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e27fb434f4ca40a79d97563be7d5b77a"
I20260807 15:11:44.570149 4626 raft_consensus.cc:3037] T 4f50f22492c6484ca89479aa0bcf9d40 P c64ad608952248c5be9d951d5be7a51d [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:11:44.570155 4250 raft_consensus.cc:3037] T 4f50f22492c6484ca89479aa0bcf9d40 P e27fb434f4ca40a79d97563be7d5b77a [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:11:44.570858 4626 raft_consensus.cc:2454] T 4f50f22492c6484ca89479aa0bcf9d40 P c64ad608952248c5be9d951d5be7a51d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0c16d4b5aeb1494a9b8dac028113f72f in term 1.
I20260807 15:11:44.570871 4250 raft_consensus.cc:2454] T 4f50f22492c6484ca89479aa0bcf9d40 P e27fb434f4ca40a79d97563be7d5b77a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0c16d4b5aeb1494a9b8dac028113f72f in term 1.
I20260807 15:11:44.571012 4352 leader_election.cc:302] T 4f50f22492c6484ca89479aa0bcf9d40 P 0c16d4b5aeb1494a9b8dac028113f72f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0c16d4b5aeb1494a9b8dac028113f72f, c64ad608952248c5be9d951d5be7a51d; no voters:
I20260807 15:11:44.571106 4729 raft_consensus.cc:2781] T 4f50f22492c6484ca89479aa0bcf9d40 P 0c16d4b5aeb1494a9b8dac028113f72f [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:11:44.571267 4729 raft_consensus.cc:686] T 4f50f22492c6484ca89479aa0bcf9d40 P 0c16d4b5aeb1494a9b8dac028113f72f [term 1 LEADER]: Becoming Leader. State: Replica: 0c16d4b5aeb1494a9b8dac028113f72f, State: Running, Role: LEADER
I20260807 15:11:44.571457 4729 consensus_queue.cc:237] T 4f50f22492c6484ca89479aa0bcf9d40 P 0c16d4b5aeb1494a9b8dac028113f72f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e27fb434f4ca40a79d97563be7d5b77a" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 36903 } } peers { permanent_uuid: "c64ad608952248c5be9d951d5be7a51d" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 39501 } } peers { permanent_uuid: "0c16d4b5aeb1494a9b8dac028113f72f" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 37305 } }
I20260807 15:11:44.572446 4041 catalog_manager.cc:5495] T 4f50f22492c6484ca89479aa0bcf9d40 P 0c16d4b5aeb1494a9b8dac028113f72f reported cstate change: term changed from 0 to 1, leader changed from <none> to 0c16d4b5aeb1494a9b8dac028113f72f (127.29.194.2). New cstate: current_term: 1 leader_uuid: "0c16d4b5aeb1494a9b8dac028113f72f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e27fb434f4ca40a79d97563be7d5b77a" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 36903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c64ad608952248c5be9d951d5be7a51d" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 39501 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0c16d4b5aeb1494a9b8dac028113f72f" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 37305 } health_report { overall_health: HEALTHY } } }
I20260807 15:11:44.579838 4713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:44.577048 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36792 (local address 127.29.194.2:37305)
0807 15:11:44.577104 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:44.577106 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:44.577115 (+ 9us) server_negotiation.cc:407] Connection header received
0807 15:11:44.577149 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:44.577149 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:44.577167 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:44.577230 (+ 63us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 15:11:44.577317 (+ 87us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:44.578052 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:44.578923 (+ 871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:44.579132 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:44.579220 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:44.579433 (+ 213us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 15:11:44.579577 (+ 144us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 15:11:44.579647 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0807 15:11:44.579731 (+ 84us) server_negotiation.cc:299] Negotiation successful
0807 15:11:44.579753 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260807 15:11:44.582571 4626 raft_consensus.cc:1261] T 4f50f22492c6484ca89479aa0bcf9d40 P c64ad608952248c5be9d951d5be7a51d [term 1 FOLLOWER]: Refusing update from remote peer 0c16d4b5aeb1494a9b8dac028113f72f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:11:44.582682 4250 raft_consensus.cc:1261] T 4f50f22492c6484ca89479aa0bcf9d40 P e27fb434f4ca40a79d97563be7d5b77a [term 1 FOLLOWER]: Refusing update from remote peer 0c16d4b5aeb1494a9b8dac028113f72f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:11:44.583091 4729 consensus_queue.cc:1035] T 4f50f22492c6484ca89479aa0bcf9d40 P 0c16d4b5aeb1494a9b8dac028113f72f [LEADER]: Connected to new peer: Peer: permanent_uuid: "c64ad608952248c5be9d951d5be7a51d" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 39501 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:11:44.583305 4733 consensus_queue.cc:1035] T 4f50f22492c6484ca89479aa0bcf9d40 P 0c16d4b5aeb1494a9b8dac028113f72f [LEADER]: Connected to new peer: Peer: permanent_uuid: "e27fb434f4ca40a79d97563be7d5b77a" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 36903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:11:44.586620 4740 mvcc.cc:205] Tried to move back new op lower bound from 7315929106765836288 to 7315929106725216256. Current Snapshot: MvccSnapshot[applied={T|T < 7315929106765836288}]
I20260807 15:11:44.586935 4738 mvcc.cc:205] Tried to move back new op lower bound from 7315929106765836288 to 7315929106725216256. Current Snapshot: MvccSnapshot[applied={T|T < 7315929106765836288}]
I20260807 15:11:44.589268 4041 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36808:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 15:11:44.589380 4041 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36808:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 15:11:44.590384 4041 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8b96e2433d2b4768916a8bd803d35282: Sending DeleteTablet for 3 replicas of tablet 4f50f22492c6484ca89479aa0bcf9d40
I20260807 15:11:44.590971 4218 tablet_service.cc:1514] Processing DeleteTablet for tablet 4f50f22492c6484ca89479aa0bcf9d40 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:11:44 UTC) from {username='kudu', principal='kudu/127.29.194.62@KRBTEST.COM'} at 127.0.0.1:45464
I20260807 15:11:44.591037 4410 tablet_service.cc:1514] Processing DeleteTablet for tablet 4f50f22492c6484ca89479aa0bcf9d40 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:11:44 UTC) from {username='kudu', principal='kudu/127.29.194.62@KRBTEST.COM'} at 127.0.0.1:36790
I20260807 15:11:44.591269 4752 tablet_replica.cc:330] T 4f50f22492c6484ca89479aa0bcf9d40 P e27fb434f4ca40a79d97563be7d5b77a: stopping tablet replica
I20260807 15:11:44.591264 4596 tablet_service.cc:1514] Processing DeleteTablet for tablet 4f50f22492c6484ca89479aa0bcf9d40 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:11:44 UTC) from {username='kudu', principal='kudu/127.29.194.62@KRBTEST.COM'} at 127.0.0.1:55510
I20260807 15:11:44.591390 4752 raft_consensus.cc:2229] T 4f50f22492c6484ca89479aa0bcf9d40 P e27fb434f4ca40a79d97563be7d5b77a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 15:11:44.591512 4753 tablet_replica.cc:330] T 4f50f22492c6484ca89479aa0bcf9d40 P 0c16d4b5aeb1494a9b8dac028113f72f: stopping tablet replica
I20260807 15:11:44.591526 4752 raft_consensus.cc:2258] T 4f50f22492c6484ca89479aa0bcf9d40 P e27fb434f4ca40a79d97563be7d5b77a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:11:44.591611 4753 raft_consensus.cc:2229] T 4f50f22492c6484ca89479aa0bcf9d40 P 0c16d4b5aeb1494a9b8dac028113f72f [term 1 LEADER]: Raft consensus shutting down.
I20260807 15:11:44.591831 4753 raft_consensus.cc:2258] T 4f50f22492c6484ca89479aa0bcf9d40 P 0c16d4b5aeb1494a9b8dac028113f72f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:11:44.591869 30472 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7fZSIL/build/release/bin/kudu with pid 4129
I20260807 15:11:44.592036 4752 ts_tablet_manager.cc:1905] T 4f50f22492c6484ca89479aa0bcf9d40 P e27fb434f4ca40a79d97563be7d5b77a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:11:44.592271 4753 ts_tablet_manager.cc:1905] T 4f50f22492c6484ca89479aa0bcf9d40 P 0c16d4b5aeb1494a9b8dac028113f72f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:11:44.594307 4753 ts_tablet_manager.cc:1918] T 4f50f22492c6484ca89479aa0bcf9d40 P 0c16d4b5aeb1494a9b8dac028113f72f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 15:11:44.594393 4753 log.cc:1196] T 4f50f22492c6484ca89479aa0bcf9d40 P 0c16d4b5aeb1494a9b8dac028113f72f: Deleting WAL directory at /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115480851507-30472-0/minicluster-data/ts-1/wal/wals/4f50f22492c6484ca89479aa0bcf9d40
I20260807 15:11:44.594744 4753 ts_tablet_manager.cc:1939] T 4f50f22492c6484ca89479aa0bcf9d40 P 0c16d4b5aeb1494a9b8dac028113f72f: Deleting consensus metadata
I20260807 15:11:44.598228 4010 catalog_manager.cc:4841] TS 0c16d4b5aeb1494a9b8dac028113f72f (127.29.194.2:37305): tablet 4f50f22492c6484ca89479aa0bcf9d40 (table test-table [id=231cd01cfaea43f996182d43bff33058]) successfully deleted
I20260807 15:11:44.598738 4755 tablet_replica.cc:330] T 4f50f22492c6484ca89479aa0bcf9d40 P c64ad608952248c5be9d951d5be7a51d: stopping tablet replica
I20260807 15:11:44.598971 4755 raft_consensus.cc:2229] T 4f50f22492c6484ca89479aa0bcf9d40 P c64ad608952248c5be9d951d5be7a51d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 15:11:44.599143 4755 raft_consensus.cc:2258] T 4f50f22492c6484ca89479aa0bcf9d40 P c64ad608952248c5be9d951d5be7a51d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:11:44.599740 4755 ts_tablet_manager.cc:1905] T 4f50f22492c6484ca89479aa0bcf9d40 P c64ad608952248c5be9d951d5be7a51d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:11:44.601826 4755 ts_tablet_manager.cc:1918] T 4f50f22492c6484ca89479aa0bcf9d40 P c64ad608952248c5be9d951d5be7a51d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 15:11:44.601954 4755 log.cc:1196] T 4f50f22492c6484ca89479aa0bcf9d40 P c64ad608952248c5be9d951d5be7a51d: Deleting WAL directory at /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115480851507-30472-0/minicluster-data/ts-2/wal/wals/4f50f22492c6484ca89479aa0bcf9d40
W20260807 15:11:44.602136 4011 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.29.194.1:36903 (error 108)
I20260807 15:11:44.602263 4755 ts_tablet_manager.cc:1939] T 4f50f22492c6484ca89479aa0bcf9d40 P c64ad608952248c5be9d951d5be7a51d: Deleting consensus metadata
I20260807 15:11:44.602763 4714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:44.602625 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.1:36903 (local address 127.0.0.1:45472)
0807 15:11:44.602684 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0807 15:11:44.602731 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.194.1:36903: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260807 15:11:44.602877 4011 catalog_manager.cc:4584] TS e27fb434f4ca40a79d97563be7d5b77a (127.29.194.1:36903): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 4f50f22492c6484ca89479aa0bcf9d40: Network error: Client connection negotiation failed: client connection to 127.29.194.1:36903: connect: Connection refused (error 111)
I20260807 15:11:44.602962 4015 catalog_manager.cc:4841] TS c64ad608952248c5be9d951d5be7a51d (127.29.194.3:39501): tablet 4f50f22492c6484ca89479aa0bcf9d40 (table test-table [id=231cd01cfaea43f996182d43bff33058]) successfully deleted
I20260807 15:11:44.603148 30472 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7fZSIL/build/release/bin/kudu with pid 4320
I20260807 15:11:44.626062 30472 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7fZSIL/build/release/bin/kudu with pid 4511
I20260807 15:11:44.639213 30472 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7fZSIL/build/release/bin/kudu with pid 3986
2026-08-07T15:11:44Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (2322 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-07T15:11:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T15:11:44Z Disabled control of system clock
I20260807 15:11:44.680017 30472 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7fZSIL/build/release/bin/kudu
/tmp/dist-test-task7fZSIL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115480851507-30472-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115480851507-30472-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115480851507-30472-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115480851507-30472-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.194.62:39045
--webserver_interface=127.29.194.62
--webserver_port=0
--builtin_ntp_servers=127.29.194.20:44457
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.194.62:39045
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260807 15:11:44.766126 4761 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:11:44.766361 4761 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:11:44.766386 4761 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:11:44.767601 4761 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:11:44.767640 4761 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:11:44.767652 4761 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:11:44.767663 4761 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:11:44.767674 4761 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:11:44.769058 4761 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.194.20:44457
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115480851507-30472-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115480851507-30472-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.194.62:39045
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.194.62:39045
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115480851507-30472-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.194.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4761
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115480851507-30472-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:11:44.769289 4761 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:11:44.769587 4761 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:11:44.775104 4767 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:11:44.775152 4769 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:11:44.775342 4761 server_base.cc:1034] running on GCE node
W20260807 15:11:44.775566 4772 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:11:44.776297 4761 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:11:44.776650 4761 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:11:44.777839 4761 hybrid_clock.cc:648] HybridClock initialized: now 1786115504777810 us; error 52 us; skew 500 ppm
I20260807 15:11:44.779541 4761 webserver.cc:460] Webserver started at http://127.29.194.62:45301/ using document root <none> and password file <none>
I20260807 15:11:44.779786 4761 fs_manager.cc:359] Metadata directory not provided
I20260807 15:11:44.779841 4761 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:11:44.779983 4761 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:11:44.781126 4761 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115480851507-30472-0/minicluster-data/master-0/data/instance:
uuid: "29c3551afcdf4c949e23f7e93bb8aa3c"
format_stamp: "Formatted at 2026-08-07 15:11:44 on dist-test-slave-42z9"
server_key: "ad0671abc716a607071c233eb03ddf84"
server_key_iv: "2bb7d7ead0198ebb83864c81c5f8bdc3"
server_key_version: "encryptionkey@0"
I20260807 15:11:44.781539 4761 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115480851507-30472-0/minicluster-data/master-0/wal/instance:
uuid: "29c3551afcdf4c949e23f7e93bb8aa3c"
format_stamp: "Formatted at 2026-08-07 15:11:44 on dist-test-slave-42z9"
server_key: "ad0671abc716a607071c233eb03ddf84"
server_key_iv: "2bb7d7ead0198ebb83864c81c5f8bdc3"
server_key_version: "encryptionkey@0"
I20260807 15:11:44.782984 4761 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260807 15:11:44.784036 4781 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:11:44.784441 4761 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 15:11:44.784583 4761 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115480851507-30472-0/minicluster-data/master-0/data,/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115480851507-30472-0/minicluster-data/master-0/wal
uuid: "29c3551afcdf4c949e23f7e93bb8aa3c"
format_stamp: "Formatted at 2026-08-07 15:11:44 on dist-test-slave-42z9"
server_key: "ad0671abc716a607071c233eb03ddf84"
server_key_iv: "2bb7d7ead0198ebb83864c81c5f8bdc3"
server_key_version: "encryptionkey@0"
I20260807 15:11:44.784695 4761 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115480851507-30472-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115480851507-30472-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115480851507-30472-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:11:44.788504 4761 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:11:44.788811 4761 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:11:44.788938 4761 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:11:44.793217 4761 rpc_server.cc:307] RPC server started. Bound to: 127.29.194.62:39045
I20260807 15:11:44.793270 4859 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.194.62:39045 every 8 connection(s)
I20260807 15:11:44.793679 4761 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115480851507-30472-0/minicluster-data/master-0/data/info.pb
I20260807 15:11:44.794653 4860 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:11:44.795207 30472 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7fZSIL/build/release/bin/kudu as pid 4761
I20260807 15:11:44.795320 30472 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115480851507-30472-0/minicluster-data/master-0/wal/instance
I20260807 15:11:44.795550 30472 external_mini_cluster.cc:1428] Setting key 872c5b81ed3c8c2d2d3609149a17f5ae
I20260807 15:11:44.798224 4860 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 29c3551afcdf4c949e23f7e93bb8aa3c: Bootstrap starting.
I20260807 15:11:44.798870 4860 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 29c3551afcdf4c949e23f7e93bb8aa3c: Neither blocks nor log segments found. Creating new log.
I20260807 15:11:44.799172 4863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:44.796367 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42238 (local address 127.29.194.62:39045)
0807 15:11:44.796578 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:44.796585 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:44.796611 (+ 26us) server_negotiation.cc:407] Connection header received
0807 15:11:44.797046 (+ 435us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:44.797052 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:44.797094 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:44.797221 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:11:44.798056 (+ 835us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:11:44.798063 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:11:44.798063 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:11:44.798071 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:11:44.798186 (+ 115us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:11:44.798205 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:11:44.798785 (+ 580us) server_negotiation.cc:1035] Waiting for connection context
0807 15:11:44.798911 (+ 126us) server_negotiation.cc:299] Negotiation successful
0807 15:11:44.798978 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141,"thread_start_us":85,"threads_started":1}
I20260807 15:11:44.799175 4860 log.cc:824] T 00000000000000000000000000000000 P 29c3551afcdf4c949e23f7e93bb8aa3c: Log is configured to *not* fsync() on all Append() calls
I20260807 15:11:44.800068 4860 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 29c3551afcdf4c949e23f7e93bb8aa3c: No bootstrap required, opened a new log
I20260807 15:11:44.802752 4860 raft_consensus.cc:348] T 00000000000000000000000000000000 P 29c3551afcdf4c949e23f7e93bb8aa3c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "29c3551afcdf4c949e23f7e93bb8aa3c" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 39045 } }
I20260807 15:11:44.802990 4860 raft_consensus.cc:374] T 00000000000000000000000000000000 P 29c3551afcdf4c949e23f7e93bb8aa3c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:11:44.803016 4860 raft_consensus.cc:729] T 00000000000000000000000000000000 P 29c3551afcdf4c949e23f7e93bb8aa3c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 29c3551afcdf4c949e23f7e93bb8aa3c, State: Initialized, Role: FOLLOWER
I20260807 15:11:44.803121 4860 consensus_queue.cc:260] T 00000000000000000000000000000000 P 29c3551afcdf4c949e23f7e93bb8aa3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "29c3551afcdf4c949e23f7e93bb8aa3c" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 39045 } }
I20260807 15:11:44.803223 4860 raft_consensus.cc:388] T 00000000000000000000000000000000 P 29c3551afcdf4c949e23f7e93bb8aa3c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:11:44.803256 4860 raft_consensus.cc:482] T 00000000000000000000000000000000 P 29c3551afcdf4c949e23f7e93bb8aa3c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:11:44.803321 4860 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 29c3551afcdf4c949e23f7e93bb8aa3c [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:11:44.804020 4860 raft_consensus.cc:504] T 00000000000000000000000000000000 P 29c3551afcdf4c949e23f7e93bb8aa3c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "29c3551afcdf4c949e23f7e93bb8aa3c" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 39045 } }
I20260807 15:11:44.804132 4860 leader_election.cc:302] T 00000000000000000000000000000000 P 29c3551afcdf4c949e23f7e93bb8aa3c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 29c3551afcdf4c949e23f7e93bb8aa3c; no voters:
I20260807 15:11:44.804313 4860 leader_election.cc:288] T 00000000000000000000000000000000 P 29c3551afcdf4c949e23f7e93bb8aa3c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 15:11:44.804584 4860 sys_catalog.cc:564] T 00000000000000000000000000000000 P 29c3551afcdf4c949e23f7e93bb8aa3c [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:11:44.804725 4868 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 29c3551afcdf4c949e23f7e93bb8aa3c [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:11:44.804899 4868 raft_consensus.cc:686] T 00000000000000000000000000000000 P 29c3551afcdf4c949e23f7e93bb8aa3c [term 1 LEADER]: Becoming Leader. State: Replica: 29c3551afcdf4c949e23f7e93bb8aa3c, State: Running, Role: LEADER
I20260807 15:11:44.805034 4868 consensus_queue.cc:237] T 00000000000000000000000000000000 P 29c3551afcdf4c949e23f7e93bb8aa3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "29c3551afcdf4c949e23f7e93bb8aa3c" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 39045 } }
I20260807 15:11:44.805569 4870 sys_catalog.cc:455] T 00000000000000000000000000000000 P 29c3551afcdf4c949e23f7e93bb8aa3c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 29c3551afcdf4c949e23f7e93bb8aa3c. Latest consensus state: current_term: 1 leader_uuid: "29c3551afcdf4c949e23f7e93bb8aa3c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "29c3551afcdf4c949e23f7e93bb8aa3c" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 39045 } } }
I20260807 15:11:44.805675 4870 sys_catalog.cc:458] T 00000000000000000000000000000000 P 29c3551afcdf4c949e23f7e93bb8aa3c [sys.catalog]: This master's current role is: LEADER
I20260807 15:11:44.805655 4869 sys_catalog.cc:455] T 00000000000000000000000000000000 P 29c3551afcdf4c949e23f7e93bb8aa3c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "29c3551afcdf4c949e23f7e93bb8aa3c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "29c3551afcdf4c949e23f7e93bb8aa3c" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 39045 } } }
I20260807 15:11:44.805751 4869 sys_catalog.cc:458] T 00000000000000000000000000000000 P 29c3551afcdf4c949e23f7e93bb8aa3c [sys.catalog]: This master's current role is: LEADER
W20260807 15:11:44.806486 4890 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 29c3551afcdf4c949e23f7e93bb8aa3c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 15:11:44.806545 4890 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 15:11:44.806614 4887 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:11:44.806785 4887 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:11:44.808333 4887 catalog_manager.cc:1347] Generated new cluster ID: 15af03af79254d709cf8794723fe6369
I20260807 15:11:44.808395 4887 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:11:44.821679 4887 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:11:44.821987 4887 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:11:44.841609 4887 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 29c3551afcdf4c949e23f7e93bb8aa3c: Generated new TSK 0
I20260807 15:11:44.841832 4887 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 15:11:44.852210 30472 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7fZSIL/build/release/bin/kudu
/tmp/dist-test-task7fZSIL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115480851507-30472-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115480851507-30472-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115480851507-30472-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115480851507-30472-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.194.1:0
--local_ip_for_outbound_sockets=127.29.194.1
--webserver_interface=127.29.194.1
--webserver_port=0
--tserver_master_addrs=127.29.194.62:39045
--builtin_ntp_servers=127.29.194.20:44457
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260807 15:11:44.945957 4895 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:11:44.946246 4895 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:11:44.946275 4895 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:11:44.948145 4895 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:11:44.948221 4895 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:11:44.948256 4895 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.194.1
I20260807 15:11:44.950187 4895 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.194.20:44457
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115480851507-30472-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115480851507-30472-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.194.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115480851507-30472-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.194.1
--webserver_port=0
--tserver_master_addrs=127.29.194.62:39045
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4895
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.194.1
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115480851507-30472-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:11:44.950496 4895 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:11:44.950829 4895 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:11:44.956501 4906 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:11:44.956566 4895 server_base.cc:1034] running on GCE node
W20260807 15:11:44.956532 4903 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:11:44.956528 4900 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:11:44.956975 4895 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:11:44.957252 4895 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:11:44.958434 4895 hybrid_clock.cc:648] HybridClock initialized: now 1786115504958413 us; error 42 us; skew 500 ppm
I20260807 15:11:44.960175 4895 webserver.cc:460] Webserver started at http://127.29.194.1:42199/ using document root <none> and password file <none>
I20260807 15:11:44.960425 4895 fs_manager.cc:359] Metadata directory not provided
I20260807 15:11:44.960489 4895 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:11:44.960623 4895 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:11:44.961822 4895 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115480851507-30472-0/minicluster-data/ts-0/data/instance:
uuid: "dea3c7495b1649968e4c4f85f46fad98"
format_stamp: "Formatted at 2026-08-07 15:11:44 on dist-test-slave-42z9"
server_key: "286e87e5374a0a6513d66decbfe79c0d"
server_key_iv: "fd41f0f568181ddfcf97adac985a847c"
server_key_version: "encryptionkey@0"
I20260807 15:11:44.962270 4895 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115480851507-30472-0/minicluster-data/ts-0/wal/instance:
uuid: "dea3c7495b1649968e4c4f85f46fad98"
format_stamp: "Formatted at 2026-08-07 15:11:44 on dist-test-slave-42z9"
server_key: "286e87e5374a0a6513d66decbfe79c0d"
server_key_iv: "fd41f0f568181ddfcf97adac985a847c"
server_key_version: "encryptionkey@0"
I20260807 15:11:44.963935 4895 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 15:11:44.965181 4914 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:11:44.965415 4895 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:11:44.965544 4895 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115480851507-30472-0/minicluster-data/ts-0/data,/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115480851507-30472-0/minicluster-data/ts-0/wal
uuid: "dea3c7495b1649968e4c4f85f46fad98"
format_stamp: "Formatted at 2026-08-07 15:11:44 on dist-test-slave-42z9"
server_key: "286e87e5374a0a6513d66decbfe79c0d"
server_key_iv: "fd41f0f568181ddfcf97adac985a847c"
server_key_version: "encryptionkey@0"
I20260807 15:11:44.965620 4895 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115480851507-30472-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115480851507-30472-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115480851507-30472-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:11:44.969209 4895 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:11:44.969551 4895 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:11:44.969674 4895 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:11:44.969871 4895 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:11:44.970168 4895 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:11:44.970201 4895 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:11:44.970222 4895 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:11:44.970233 4895 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:11:44.976019 4895 rpc_server.cc:307] RPC server started. Bound to: 127.29.194.1:40227
I20260807 15:11:44.976164 5082 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.194.1:40227 every 8 connection(s)
I20260807 15:11:44.976409 4895 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115480851507-30472-0/minicluster-data/ts-0/data/info.pb
I20260807 15:11:44.978607 30472 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7fZSIL/build/release/bin/kudu as pid 4895
I20260807 15:11:44.978847 30472 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115480851507-30472-0/minicluster-data/ts-0/wal/instance
I20260807 15:11:44.979123 30472 external_mini_cluster.cc:1428] Setting key 0244adcf1d60204f39fc47c695cdb627
I20260807 15:11:44.980381 30472 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7fZSIL/build/release/bin/kudu
/tmp/dist-test-task7fZSIL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115480851507-30472-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115480851507-30472-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115480851507-30472-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115480851507-30472-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.194.2:0
--local_ip_for_outbound_sockets=127.29.194.2
--webserver_interface=127.29.194.2
--webserver_port=0
--tserver_master_addrs=127.29.194.62:39045
--builtin_ntp_servers=127.29.194.20:44457
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260807 15:11:44.980772 4863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:44.977372 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.194.1:45423 (local address 127.29.194.62:39045)
0807 15:11:44.977435 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:44.977453 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:44.977726 (+ 273us) server_negotiation.cc:407] Connection header received
0807 15:11:44.978390 (+ 664us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:44.978392 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:44.978400 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:44.978472 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:11:44.980396 (+ 1924us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:11:44.980397 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:11:44.980398 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:11:44.980403 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:11:44.980426 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:11:44.980431 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:11:44.980518 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0807 15:11:44.980662 (+ 144us) server_negotiation.cc:299] Negotiation successful
0807 15:11:44.980680 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260807 15:11:44.980969 5087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:44.977254 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:39045 (local address 127.29.194.1:45423)
0807 15:11:44.977547 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0807 15:11:44.977591 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0807 15:11:44.978220 (+ 629us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:11:44.978580 (+ 360us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:11:44.978586 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:11:44.978600 (+ 14us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:11:44.980171 (+ 1571us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 15:11:44.980172 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:11:44.980227 (+ 55us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 15:11:44.980279 (+ 52us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:11:44.980564 (+ 285us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:11:44.980573 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:11:44.980578 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 15:11:44.980670 (+ 92us) client_negotiation.cc:240] Negotiation successful
0807 15:11:44.980753 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":79,"threads_started":1}
I20260807 15:11:44.981823 5083 heartbeater.cc:347] Connected to a master server at 127.29.194.62:39045
I20260807 15:11:44.981948 5083 heartbeater.cc:464] Registering TS with master...
I20260807 15:11:44.982132 5083 heartbeater.cc:511] Master 127.29.194.62:39045 requested a full tablet report, sending...
I20260807 15:11:44.982728 4804 ts_manager.cc:194] Registered new tserver with Master: dea3c7495b1649968e4c4f85f46fad98 (127.29.194.1:40227)
W20260807 15:11:45.075376 5088 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:11:45.075623 5088 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:11:45.075649 5088 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:11:45.076998 5088 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:11:45.077049 5088 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:11:45.077078 5088 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.194.2
I20260807 15:11:45.078632 5088 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.194.20:44457
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115480851507-30472-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115480851507-30472-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.194.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115480851507-30472-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.194.2
--webserver_port=0
--tserver_master_addrs=127.29.194.62:39045
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5088
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.194.2
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115480851507-30472-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:11:45.078871 5088 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:11:45.079150 5088 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:11:45.084853 5097 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:11:45.085534 5096 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:11:45.085565 5102 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:11:45.085642 5088 server_base.cc:1034] running on GCE node
I20260807 15:11:45.085903 5088 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:11:45.086169 5088 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:11:45.087311 5088 hybrid_clock.cc:648] HybridClock initialized: now 1786115505087297 us; error 29 us; skew 500 ppm
I20260807 15:11:45.088764 5088 webserver.cc:460] Webserver started at http://127.29.194.2:33251/ using document root <none> and password file <none>
I20260807 15:11:45.089005 5088 fs_manager.cc:359] Metadata directory not provided
I20260807 15:11:45.089058 5088 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:11:45.089191 5088 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:11:45.090617 5088 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115480851507-30472-0/minicluster-data/ts-1/data/instance:
uuid: "d185489eff86488f968d932466928c30"
format_stamp: "Formatted at 2026-08-07 15:11:45 on dist-test-slave-42z9"
server_key: "b0b7f9be15f98da1da0edc03b1893a67"
server_key_iv: "11a605bc0d6da420881bbec2009e4cdf"
server_key_version: "encryptionkey@0"
I20260807 15:11:45.090986 5088 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115480851507-30472-0/minicluster-data/ts-1/wal/instance:
uuid: "d185489eff86488f968d932466928c30"
format_stamp: "Formatted at 2026-08-07 15:11:45 on dist-test-slave-42z9"
server_key: "b0b7f9be15f98da1da0edc03b1893a67"
server_key_iv: "11a605bc0d6da420881bbec2009e4cdf"
server_key_version: "encryptionkey@0"
I20260807 15:11:45.092547 5088 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260807 15:11:45.093885 5109 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:11:45.094147 5088 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.001s
I20260807 15:11:45.094257 5088 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115480851507-30472-0/minicluster-data/ts-1/data,/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115480851507-30472-0/minicluster-data/ts-1/wal
uuid: "d185489eff86488f968d932466928c30"
format_stamp: "Formatted at 2026-08-07 15:11:45 on dist-test-slave-42z9"
server_key: "b0b7f9be15f98da1da0edc03b1893a67"
server_key_iv: "11a605bc0d6da420881bbec2009e4cdf"
server_key_version: "encryptionkey@0"
I20260807 15:11:45.094343 5088 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115480851507-30472-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115480851507-30472-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115480851507-30472-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:11:45.097960 5088 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:11:45.098253 5088 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:11:45.098564 5088 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:11:45.098935 5088 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:11:45.099404 5088 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:11:45.099445 5088 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:11:45.099478 5088 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:11:45.099505 5088 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:11:45.106956 5088 rpc_server.cc:307] RPC server started. Bound to: 127.29.194.2:36235
I20260807 15:11:45.107055 5279 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.194.2:36235 every 8 connection(s)
I20260807 15:11:45.107467 5088 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115480851507-30472-0/minicluster-data/ts-1/data/info.pb
I20260807 15:11:45.111719 4863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:45.108482 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.194.2:39163 (local address 127.29.194.62:39045)
0807 15:11:45.108560 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:45.108562 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:45.108837 (+ 275us) server_negotiation.cc:407] Connection header received
0807 15:11:45.109477 (+ 640us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:45.109479 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:45.109488 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:45.109686 (+ 198us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:11:45.111411 (+ 1725us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:11:45.111413 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:11:45.111414 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:11:45.111420 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:11:45.111482 (+ 62us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:11:45.111488 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:11:45.111561 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0807 15:11:45.111624 (+ 63us) server_negotiation.cc:299] Negotiation successful
0807 15:11:45.111639 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260807 15:11:45.111949 5282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:45.108429 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:39045 (local address 127.29.194.2:39163)
0807 15:11:45.108676 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0807 15:11:45.108725 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0807 15:11:45.109303 (+ 578us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:11:45.109849 (+ 546us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:11:45.109854 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:11:45.109865 (+ 11us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:11:45.111175 (+ 1310us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 15:11:45.111176 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:11:45.111222 (+ 46us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 15:11:45.111270 (+ 48us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:11:45.111560 (+ 290us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:11:45.111564 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:11:45.111567 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 15:11:45.111630 (+ 63us) client_negotiation.cc:240] Negotiation successful
0807 15:11:45.111701 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":122,"threads_started":1}
I20260807 15:11:45.112749 5280 heartbeater.cc:347] Connected to a master server at 127.29.194.62:39045
I20260807 15:11:45.113015 5280 heartbeater.cc:464] Registering TS with master...
I20260807 15:11:45.113237 5280 heartbeater.cc:511] Master 127.29.194.62:39045 requested a full tablet report, sending...
I20260807 15:11:45.113698 4804 ts_manager.cc:194] Registered new tserver with Master: d185489eff86488f968d932466928c30 (127.29.194.2:36235)
I20260807 15:11:45.117340 30472 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7fZSIL/build/release/bin/kudu as pid 5088
I20260807 15:11:45.117432 30472 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115480851507-30472-0/minicluster-data/ts-1/wal/instance
I20260807 15:11:45.117640 30472 external_mini_cluster.cc:1428] Setting key 9a9dd3943fd3a78bf024f6299ba3104d
I20260807 15:11:45.118695 30472 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7fZSIL/build/release/bin/kudu
/tmp/dist-test-task7fZSIL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115480851507-30472-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115480851507-30472-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115480851507-30472-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115480851507-30472-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.194.3:0
--local_ip_for_outbound_sockets=127.29.194.3
--webserver_interface=127.29.194.3
--webserver_port=0
--tserver_master_addrs=127.29.194.62:39045
--builtin_ntp_servers=127.29.194.20:44457
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260807 15:11:45.222975 5284 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:11:45.223304 5284 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:11:45.223336 5284 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:11:45.225430 5284 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:11:45.225562 5284 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:11:45.225610 5284 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.194.3
I20260807 15:11:45.228252 5284 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.194.20:44457
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115480851507-30472-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115480851507-30472-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.194.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115480851507-30472-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.194.3
--webserver_port=0
--tserver_master_addrs=127.29.194.62:39045
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5284
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.194.3
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115480851507-30472-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:11:45.228611 5284 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:11:45.228966 5284 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:11:45.234738 5297 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:11:45.234737 5293 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:11:45.234901 5291 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:11:45.235406 5284 server_base.cc:1034] running on GCE node
I20260807 15:11:45.235705 5284 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:11:45.236011 5284 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:11:45.237192 5284 hybrid_clock.cc:648] HybridClock initialized: now 1786115505237156 us; error 66 us; skew 500 ppm
I20260807 15:11:45.238765 5284 webserver.cc:460] Webserver started at http://127.29.194.3:43139/ using document root <none> and password file <none>
I20260807 15:11:45.239049 5284 fs_manager.cc:359] Metadata directory not provided
I20260807 15:11:45.239107 5284 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:11:45.239250 5284 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:11:45.240428 5284 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115480851507-30472-0/minicluster-data/ts-2/data/instance:
uuid: "cf8adb4013954bd2972e1af9c0bff9e3"
format_stamp: "Formatted at 2026-08-07 15:11:45 on dist-test-slave-42z9"
server_key: "764cd7f1216240d6cf7a5e270bf2d999"
server_key_iv: "7cc965adb772b6df66330a422fc37499"
server_key_version: "encryptionkey@0"
I20260807 15:11:45.240792 5284 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115480851507-30472-0/minicluster-data/ts-2/wal/instance:
uuid: "cf8adb4013954bd2972e1af9c0bff9e3"
format_stamp: "Formatted at 2026-08-07 15:11:45 on dist-test-slave-42z9"
server_key: "764cd7f1216240d6cf7a5e270bf2d999"
server_key_iv: "7cc965adb772b6df66330a422fc37499"
server_key_version: "encryptionkey@0"
I20260807 15:11:45.242295 5284 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260807 15:11:45.243472 5309 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:11:45.243759 5284 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 15:11:45.243863 5284 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115480851507-30472-0/minicluster-data/ts-2/data,/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115480851507-30472-0/minicluster-data/ts-2/wal
uuid: "cf8adb4013954bd2972e1af9c0bff9e3"
format_stamp: "Formatted at 2026-08-07 15:11:45 on dist-test-slave-42z9"
server_key: "764cd7f1216240d6cf7a5e270bf2d999"
server_key_iv: "7cc965adb772b6df66330a422fc37499"
server_key_version: "encryptionkey@0"
I20260807 15:11:45.243954 5284 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115480851507-30472-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115480851507-30472-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115480851507-30472-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:11:45.247591 5284 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:11:45.247884 5284 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:11:45.248014 5284 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:11:45.248214 5284 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:11:45.248541 5284 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:11:45.248579 5284 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:11:45.248608 5284 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:11:45.248632 5284 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:11:45.254777 5284 rpc_server.cc:307] RPC server started. Bound to: 127.29.194.3:41777
I20260807 15:11:45.254837 5475 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.194.3:41777 every 8 connection(s)
I20260807 15:11:45.255199 5284 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115480851507-30472-0/minicluster-data/ts-2/data/info.pb
I20260807 15:11:45.256057 30472 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7fZSIL/build/release/bin/kudu as pid 5284
I20260807 15:11:45.256178 30472 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115480851507-30472-0/minicluster-data/ts-2/wal/instance
I20260807 15:11:45.256438 30472 external_mini_cluster.cc:1428] Setting key 5c66fddb0b486afce550740d21d8f3b3
I20260807 15:11:45.258610 4863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:45.256023 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.194.3:55177 (local address 127.29.194.62:39045)
0807 15:11:45.256087 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:45.256089 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:45.256392 (+ 303us) server_negotiation.cc:407] Connection header received
0807 15:11:45.257019 (+ 627us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:45.257020 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:45.257027 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:45.257060 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:11:45.258357 (+ 1297us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:11:45.258358 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:11:45.258358 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:11:45.258363 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:11:45.258381 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:11:45.258385 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:11:45.258441 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0807 15:11:45.258518 (+ 77us) server_negotiation.cc:299] Negotiation successful
0807 15:11:45.258533 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260807 15:11:45.258816 5478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:45.255934 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:39045 (local address 127.29.194.3:55177)
0807 15:11:45.256210 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0807 15:11:45.256260 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0807 15:11:45.256925 (+ 665us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:11:45.257136 (+ 211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:11:45.257141 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:11:45.257151 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:11:45.258191 (+ 1040us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 15:11:45.258192 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:11:45.258226 (+ 34us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 15:11:45.258260 (+ 34us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:11:45.258449 (+ 189us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:11:45.258454 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:11:45.258458 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 15:11:45.258532 (+ 74us) client_negotiation.cc:240] Negotiation successful
0807 15:11:45.258600 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":92,"threads_started":1}
I20260807 15:11:45.259241 5476 heartbeater.cc:347] Connected to a master server at 127.29.194.62:39045
I20260807 15:11:45.259343 5476 heartbeater.cc:464] Registering TS with master...
I20260807 15:11:45.259502 5476 heartbeater.cc:511] Master 127.29.194.62:39045 requested a full tablet report, sending...
I20260807 15:11:45.259920 4804 ts_manager.cc:194] Registered new tserver with Master: cf8adb4013954bd2972e1af9c0bff9e3 (127.29.194.3:41777)
I20260807 15:11:45.269146 30472 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 15:11:45.271759 4863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:45.270634 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42248 (local address 127.29.194.62:39045)
0807 15:11:45.270699 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:45.270701 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:45.270800 (+ 99us) server_negotiation.cc:407] Connection header received
0807 15:11:45.270867 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:45.270868 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:45.270874 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:45.270924 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:11:45.271544 (+ 620us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:11:45.271545 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:11:45.271546 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:11:45.271550 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:11:45.271565 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:11:45.271569 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:11:45.271613 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0807 15:11:45.271664 (+ 51us) server_negotiation.cc:299] Negotiation successful
0807 15:11:45.271679 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260807 15:11:45.273893 4804 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:42248:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 15:11:45.275532 4804 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 15:11:45.281772 5492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:45.279739 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.2:36235 (local address 127.0.0.1:48312)
0807 15:11:45.280098 (+ 359us) negotiation.cc:107] Waiting for socket to connect
0807 15:11:45.280107 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0807 15:11:45.280149 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:11:45.280421 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:11:45.280422 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:11:45.280425 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:11:45.281323 (+ 898us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 15:11:45.281323 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:11:45.281332 (+ 9us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 15:11:45.281345 (+ 13us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:11:45.281595 (+ 250us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:11:45.281601 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:11:45.281604 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 15:11:45.281669 (+ 65us) client_negotiation.cc:240] Negotiation successful
0807 15:11:45.281684 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":341,"thread_start_us":111,"threads_started":1}
I20260807 15:11:45.282097 5493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:45.279914 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48312 (local address 127.29.194.2:36235)
0807 15:11:45.280250 (+ 336us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:45.280254 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:45.280262 (+ 8us) server_negotiation.cc:407] Connection header received
0807 15:11:45.280288 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:45.280290 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:45.280298 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:45.280413 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:11:45.281433 (+ 1020us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:11:45.281468 (+ 35us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:11:45.281469 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:11:45.281473 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:11:45.281509 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:11:45.281529 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:11:45.281583 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0807 15:11:45.281998 (+ 415us) server_negotiation.cc:299] Negotiation successful
0807 15:11:45.282015 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":303,"thread_start_us":93,"threads_started":1}
I20260807 15:11:45.282130 5487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:45.279276 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60518 (local address 127.29.194.1:40227)
0807 15:11:45.279969 (+ 693us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:45.279974 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:45.279989 (+ 15us) server_negotiation.cc:407] Connection header received
0807 15:11:45.280023 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:45.280026 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:45.280037 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:45.280436 (+ 399us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:11:45.281433 (+ 997us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:11:45.281479 (+ 46us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:11:45.281480 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:11:45.281487 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:11:45.281547 (+ 60us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:11:45.281589 (+ 42us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:11:45.281986 (+ 397us) server_negotiation.cc:1035] Waiting for connection context
0807 15:11:45.282020 (+ 34us) server_negotiation.cc:299] Negotiation successful
0807 15:11:45.282041 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":263,"thread_start_us":162,"threads_started":1}
I20260807 15:11:45.282330 5490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:45.279706 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54668 (local address 127.29.194.3:41777)
0807 15:11:45.279986 (+ 280us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:45.279991 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:45.280002 (+ 11us) server_negotiation.cc:407] Connection header received
0807 15:11:45.280030 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:45.280032 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:45.280042 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:45.280434 (+ 392us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:11:45.281433 (+ 999us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:11:45.281484 (+ 51us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:11:45.281485 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:11:45.281491 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:11:45.281544 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:11:45.281590 (+ 46us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:11:45.281670 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0807 15:11:45.281835 (+ 165us) server_negotiation.cc:299] Negotiation successful
0807 15:11:45.281868 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":214,"thread_start_us":142,"threads_started":1}
I20260807 15:11:45.282399 5486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:45.279096 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.1:40227 (local address 127.0.0.1:60518)
0807 15:11:45.279364 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0807 15:11:45.279416 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0807 15:11:45.279580 (+ 164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:11:45.280164 (+ 584us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:11:45.280172 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:11:45.280178 (+ 6us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:11:45.281157 (+ 979us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 15:11:45.281157 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:11:45.281176 (+ 19us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 15:11:45.281204 (+ 28us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:11:45.281686 (+ 482us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:11:45.281687 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:11:45.281688 (+ 1us) client_negotiation.cc:769] Sending connection context
0807 15:11:45.281723 (+ 35us) client_negotiation.cc:240] Negotiation successful
0807 15:11:45.281850 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":211,"thread_start_us":133,"threads_started":1}
I20260807 15:11:45.281831 5488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:45.279442 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.3:41777 (local address 127.0.0.1:54668)
0807 15:11:45.279695 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0807 15:11:45.279707 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 15:11:45.279741 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:11:45.280164 (+ 423us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:11:45.280171 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:11:45.280178 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:11:45.281204 (+ 1026us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 15:11:45.281204 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:11:45.281209 (+ 5us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 15:11:45.281217 (+ 8us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:11:45.281660 (+ 443us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:11:45.281661 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:11:45.281662 (+ 1us) client_negotiation.cc:769] Sending connection context
0807 15:11:45.281728 (+ 66us) client_negotiation.cc:240] Negotiation successful
0807 15:11:45.281740 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":226,"thread_start_us":87,"threads_started":1}
I20260807 15:11:45.282843 5187 tablet_service.cc:1467] Processing CreateTablet for tablet 1209f505a81349b3bfa86cffb317a288 (DEFAULT_TABLE table=test-table [id=7599929436e44a6ca9fe498a8dbc85eb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:11:45.283140 5374 tablet_service.cc:1467] Processing CreateTablet for tablet 1209f505a81349b3bfa86cffb317a288 (DEFAULT_TABLE table=test-table [id=7599929436e44a6ca9fe498a8dbc85eb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:11:45.283286 5187 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1209f505a81349b3bfa86cffb317a288. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:11:45.283198 4984 tablet_service.cc:1467] Processing CreateTablet for tablet 1209f505a81349b3bfa86cffb317a288 (DEFAULT_TABLE table=test-table [id=7599929436e44a6ca9fe498a8dbc85eb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:11:45.283481 5374 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1209f505a81349b3bfa86cffb317a288. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:11:45.283792 4984 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1209f505a81349b3bfa86cffb317a288. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:11:45.286435 5494 tablet_bootstrap.cc:492] T 1209f505a81349b3bfa86cffb317a288 P cf8adb4013954bd2972e1af9c0bff9e3: Bootstrap starting.
I20260807 15:11:45.287021 5495 tablet_bootstrap.cc:492] T 1209f505a81349b3bfa86cffb317a288 P d185489eff86488f968d932466928c30: Bootstrap starting.
I20260807 15:11:45.287407 5494 tablet_bootstrap.cc:654] T 1209f505a81349b3bfa86cffb317a288 P cf8adb4013954bd2972e1af9c0bff9e3: Neither blocks nor log segments found. Creating new log.
I20260807 15:11:45.287396 5497 tablet_bootstrap.cc:492] T 1209f505a81349b3bfa86cffb317a288 P dea3c7495b1649968e4c4f85f46fad98: Bootstrap starting.
I20260807 15:11:45.287709 5495 tablet_bootstrap.cc:654] T 1209f505a81349b3bfa86cffb317a288 P d185489eff86488f968d932466928c30: Neither blocks nor log segments found. Creating new log.
I20260807 15:11:45.287909 5494 log.cc:824] T 1209f505a81349b3bfa86cffb317a288 P cf8adb4013954bd2972e1af9c0bff9e3: Log is configured to *not* fsync() on all Append() calls
I20260807 15:11:45.288010 5497 tablet_bootstrap.cc:654] T 1209f505a81349b3bfa86cffb317a288 P dea3c7495b1649968e4c4f85f46fad98: Neither blocks nor log segments found. Creating new log.
I20260807 15:11:45.288060 5495 log.cc:824] T 1209f505a81349b3bfa86cffb317a288 P d185489eff86488f968d932466928c30: Log is configured to *not* fsync() on all Append() calls
I20260807 15:11:45.288300 5497 log.cc:824] T 1209f505a81349b3bfa86cffb317a288 P dea3c7495b1649968e4c4f85f46fad98: Log is configured to *not* fsync() on all Append() calls
I20260807 15:11:45.288887 5495 tablet_bootstrap.cc:492] T 1209f505a81349b3bfa86cffb317a288 P d185489eff86488f968d932466928c30: No bootstrap required, opened a new log
I20260807 15:11:45.288952 5495 ts_tablet_manager.cc:1397] T 1209f505a81349b3bfa86cffb317a288 P d185489eff86488f968d932466928c30: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 15:11:45.289060 5497 tablet_bootstrap.cc:492] T 1209f505a81349b3bfa86cffb317a288 P dea3c7495b1649968e4c4f85f46fad98: No bootstrap required, opened a new log
I20260807 15:11:45.289129 5497 ts_tablet_manager.cc:1397] T 1209f505a81349b3bfa86cffb317a288 P dea3c7495b1649968e4c4f85f46fad98: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 15:11:45.289819 5494 tablet_bootstrap.cc:492] T 1209f505a81349b3bfa86cffb317a288 P cf8adb4013954bd2972e1af9c0bff9e3: No bootstrap required, opened a new log
I20260807 15:11:45.289920 5494 ts_tablet_manager.cc:1397] T 1209f505a81349b3bfa86cffb317a288 P cf8adb4013954bd2972e1af9c0bff9e3: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260807 15:11:45.291330 5497 raft_consensus.cc:348] T 1209f505a81349b3bfa86cffb317a288 P dea3c7495b1649968e4c4f85f46fad98 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dea3c7495b1649968e4c4f85f46fad98" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 40227 } } peers { permanent_uuid: "cf8adb4013954bd2972e1af9c0bff9e3" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 41777 } } peers { permanent_uuid: "d185489eff86488f968d932466928c30" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 36235 } }
I20260807 15:11:45.291481 5497 raft_consensus.cc:374] T 1209f505a81349b3bfa86cffb317a288 P dea3c7495b1649968e4c4f85f46fad98 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:11:45.291514 5497 raft_consensus.cc:729] T 1209f505a81349b3bfa86cffb317a288 P dea3c7495b1649968e4c4f85f46fad98 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dea3c7495b1649968e4c4f85f46fad98, State: Initialized, Role: FOLLOWER
I20260807 15:11:45.291481 5495 raft_consensus.cc:348] T 1209f505a81349b3bfa86cffb317a288 P d185489eff86488f968d932466928c30 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dea3c7495b1649968e4c4f85f46fad98" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 40227 } } peers { permanent_uuid: "cf8adb4013954bd2972e1af9c0bff9e3" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 41777 } } peers { permanent_uuid: "d185489eff86488f968d932466928c30" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 36235 } }
I20260807 15:11:45.291646 5495 raft_consensus.cc:374] T 1209f505a81349b3bfa86cffb317a288 P d185489eff86488f968d932466928c30 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:11:45.291656 5497 consensus_queue.cc:260] T 1209f505a81349b3bfa86cffb317a288 P dea3c7495b1649968e4c4f85f46fad98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dea3c7495b1649968e4c4f85f46fad98" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 40227 } } peers { permanent_uuid: "cf8adb4013954bd2972e1af9c0bff9e3" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 41777 } } peers { permanent_uuid: "d185489eff86488f968d932466928c30" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 36235 } }
I20260807 15:11:45.291695 5495 raft_consensus.cc:729] T 1209f505a81349b3bfa86cffb317a288 P d185489eff86488f968d932466928c30 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d185489eff86488f968d932466928c30, State: Initialized, Role: FOLLOWER
I20260807 15:11:45.291814 5495 consensus_queue.cc:260] T 1209f505a81349b3bfa86cffb317a288 P d185489eff86488f968d932466928c30 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dea3c7495b1649968e4c4f85f46fad98" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 40227 } } peers { permanent_uuid: "cf8adb4013954bd2972e1af9c0bff9e3" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 41777 } } peers { permanent_uuid: "d185489eff86488f968d932466928c30" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 36235 } }
I20260807 15:11:45.291983 5497 ts_tablet_manager.cc:1428] T 1209f505a81349b3bfa86cffb317a288 P dea3c7495b1649968e4c4f85f46fad98: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260807 15:11:45.292047 5083 heartbeater.cc:503] Master 127.29.194.62:39045 was elected leader, sending a full tablet report...
I20260807 15:11:45.292104 5495 ts_tablet_manager.cc:1428] T 1209f505a81349b3bfa86cffb317a288 P d185489eff86488f968d932466928c30: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260807 15:11:45.292169 5280 heartbeater.cc:503] Master 127.29.194.62:39045 was elected leader, sending a full tablet report...
I20260807 15:11:45.292492 5494 raft_consensus.cc:348] T 1209f505a81349b3bfa86cffb317a288 P cf8adb4013954bd2972e1af9c0bff9e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dea3c7495b1649968e4c4f85f46fad98" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 40227 } } peers { permanent_uuid: "cf8adb4013954bd2972e1af9c0bff9e3" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 41777 } } peers { permanent_uuid: "d185489eff86488f968d932466928c30" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 36235 } }
I20260807 15:11:45.292663 5494 raft_consensus.cc:374] T 1209f505a81349b3bfa86cffb317a288 P cf8adb4013954bd2972e1af9c0bff9e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:11:45.292699 5494 raft_consensus.cc:729] T 1209f505a81349b3bfa86cffb317a288 P cf8adb4013954bd2972e1af9c0bff9e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf8adb4013954bd2972e1af9c0bff9e3, State: Initialized, Role: FOLLOWER
I20260807 15:11:45.292829 5494 consensus_queue.cc:260] T 1209f505a81349b3bfa86cffb317a288 P cf8adb4013954bd2972e1af9c0bff9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dea3c7495b1649968e4c4f85f46fad98" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 40227 } } peers { permanent_uuid: "cf8adb4013954bd2972e1af9c0bff9e3" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 41777 } } peers { permanent_uuid: "d185489eff86488f968d932466928c30" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 36235 } }
I20260807 15:11:45.293092 5494 ts_tablet_manager.cc:1428] T 1209f505a81349b3bfa86cffb317a288 P cf8adb4013954bd2972e1af9c0bff9e3: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260807 15:11:45.293231 5476 heartbeater.cc:503] Master 127.29.194.62:39045 was elected leader, sending a full tablet report...
I20260807 15:11:45.343259 5505 raft_consensus.cc:482] T 1209f505a81349b3bfa86cffb317a288 P cf8adb4013954bd2972e1af9c0bff9e3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 15:11:45.343437 5505 raft_consensus.cc:504] T 1209f505a81349b3bfa86cffb317a288 P cf8adb4013954bd2972e1af9c0bff9e3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dea3c7495b1649968e4c4f85f46fad98" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 40227 } } peers { permanent_uuid: "cf8adb4013954bd2972e1af9c0bff9e3" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 41777 } } peers { permanent_uuid: "d185489eff86488f968d932466928c30" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 36235 } }
I20260807 15:11:45.343777 5505 leader_election.cc:288] T 1209f505a81349b3bfa86cffb317a288 P cf8adb4013954bd2972e1af9c0bff9e3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers dea3c7495b1649968e4c4f85f46fad98 (127.29.194.1:40227), d185489eff86488f968d932466928c30 (127.29.194.2:36235)
I20260807 15:11:45.345146 5478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:45.344008 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.1:40227 (local address 127.29.194.3:38387)
0807 15:11:45.344064 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0807 15:11:45.344077 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 15:11:45.344135 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:11:45.344311 (+ 176us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:11:45.344312 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:11:45.344316 (+ 4us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:11:45.344769 (+ 453us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 15:11:45.344770 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:11:45.344779 (+ 9us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 15:11:45.344793 (+ 14us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:11:45.345011 (+ 218us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:11:45.345011 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:11:45.345012 (+ 1us) client_negotiation.cc:769] Sending connection context
0807 15:11:45.345059 (+ 47us) client_negotiation.cc:240] Negotiation successful
0807 15:11:45.345075 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32}
I20260807 15:11:45.345146 5487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:45.344112 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.194.3:38387 (local address 127.29.194.1:40227)
0807 15:11:45.344169 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:45.344171 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:45.344182 (+ 11us) server_negotiation.cc:407] Connection header received
0807 15:11:45.344215 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:45.344216 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:45.344222 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:45.344301 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:11:45.344900 (+ 599us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:11:45.344901 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:11:45.344901 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:11:45.344907 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:11:45.344924 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:11:45.344929 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:11:45.345008 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0807 15:11:45.345061 (+ 53us) server_negotiation.cc:299] Negotiation successful
0807 15:11:45.345074 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260807 15:11:45.345546 5507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:45.344124 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.2:36235 (local address 127.29.194.3:53619)
0807 15:11:45.344320 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0807 15:11:45.344328 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0807 15:11:45.344384 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:11:45.344762 (+ 378us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:11:45.344762 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:11:45.344764 (+ 2us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:11:45.345176 (+ 412us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 15:11:45.345177 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:11:45.345184 (+ 7us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 15:11:45.345196 (+ 12us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:11:45.345388 (+ 192us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:11:45.345389 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:11:45.345389 (+ 0us) client_negotiation.cc:769] Sending connection context
0807 15:11:45.345457 (+ 68us) client_negotiation.cc:240] Negotiation successful
0807 15:11:45.345471 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":112,"threads_started":1}
I20260807 15:11:45.345561 5493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:45.344219 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.194.3:53619 (local address 127.29.194.2:36235)
0807 15:11:45.344290 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:45.344292 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:45.344384 (+ 92us) server_negotiation.cc:407] Connection header received
0807 15:11:45.344437 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:45.344438 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:45.344443 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:45.344490 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:11:45.345288 (+ 798us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:11:45.345290 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:11:45.345290 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:11:45.345295 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:11:45.345310 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:11:45.345314 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:11:45.345345 (+ 31us) server_negotiation.cc:1035] Waiting for connection context
0807 15:11:45.345479 (+ 134us) server_negotiation.cc:299] Negotiation successful
0807 15:11:45.345494 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260807 15:11:45.345582 5017 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1209f505a81349b3bfa86cffb317a288" candidate_uuid: "cf8adb4013954bd2972e1af9c0bff9e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dea3c7495b1649968e4c4f85f46fad98" is_pre_election: true
I20260807 15:11:45.345741 5017 raft_consensus.cc:2454] T 1209f505a81349b3bfa86cffb317a288 P dea3c7495b1649968e4c4f85f46fad98 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf8adb4013954bd2972e1af9c0bff9e3 in term 0.
I20260807 15:11:45.345868 5214 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1209f505a81349b3bfa86cffb317a288" candidate_uuid: "cf8adb4013954bd2972e1af9c0bff9e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d185489eff86488f968d932466928c30" is_pre_election: true
I20260807 15:11:45.345986 5214 raft_consensus.cc:2454] T 1209f505a81349b3bfa86cffb317a288 P d185489eff86488f968d932466928c30 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf8adb4013954bd2972e1af9c0bff9e3 in term 0.
I20260807 15:11:45.346010 5313 leader_election.cc:302] T 1209f505a81349b3bfa86cffb317a288 P cf8adb4013954bd2972e1af9c0bff9e3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cf8adb4013954bd2972e1af9c0bff9e3, dea3c7495b1649968e4c4f85f46fad98; no voters:
I20260807 15:11:45.346144 5505 raft_consensus.cc:2781] T 1209f505a81349b3bfa86cffb317a288 P cf8adb4013954bd2972e1af9c0bff9e3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 15:11:45.346225 5505 raft_consensus.cc:482] T 1209f505a81349b3bfa86cffb317a288 P cf8adb4013954bd2972e1af9c0bff9e3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 15:11:45.346256 5505 raft_consensus.cc:3037] T 1209f505a81349b3bfa86cffb317a288 P cf8adb4013954bd2972e1af9c0bff9e3 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:11:45.346997 5505 raft_consensus.cc:504] T 1209f505a81349b3bfa86cffb317a288 P cf8adb4013954bd2972e1af9c0bff9e3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dea3c7495b1649968e4c4f85f46fad98" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 40227 } } peers { permanent_uuid: "cf8adb4013954bd2972e1af9c0bff9e3" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 41777 } } peers { permanent_uuid: "d185489eff86488f968d932466928c30" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 36235 } }
I20260807 15:11:45.347107 5505 leader_election.cc:288] T 1209f505a81349b3bfa86cffb317a288 P cf8adb4013954bd2972e1af9c0bff9e3 [CANDIDATE]: Term 1 election: Requested vote from peers dea3c7495b1649968e4c4f85f46fad98 (127.29.194.1:40227), d185489eff86488f968d932466928c30 (127.29.194.2:36235)
I20260807 15:11:45.347285 5017 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1209f505a81349b3bfa86cffb317a288" candidate_uuid: "cf8adb4013954bd2972e1af9c0bff9e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dea3c7495b1649968e4c4f85f46fad98"
I20260807 15:11:45.347333 5214 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1209f505a81349b3bfa86cffb317a288" candidate_uuid: "cf8adb4013954bd2972e1af9c0bff9e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d185489eff86488f968d932466928c30"
I20260807 15:11:45.347374 5017 raft_consensus.cc:3037] T 1209f505a81349b3bfa86cffb317a288 P dea3c7495b1649968e4c4f85f46fad98 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:11:45.347406 5214 raft_consensus.cc:3037] T 1209f505a81349b3bfa86cffb317a288 P d185489eff86488f968d932466928c30 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:11:45.348244 5017 raft_consensus.cc:2454] T 1209f505a81349b3bfa86cffb317a288 P dea3c7495b1649968e4c4f85f46fad98 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf8adb4013954bd2972e1af9c0bff9e3 in term 1.
I20260807 15:11:45.348244 5214 raft_consensus.cc:2454] T 1209f505a81349b3bfa86cffb317a288 P d185489eff86488f968d932466928c30 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf8adb4013954bd2972e1af9c0bff9e3 in term 1.
I20260807 15:11:45.348451 5313 leader_election.cc:302] T 1209f505a81349b3bfa86cffb317a288 P cf8adb4013954bd2972e1af9c0bff9e3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cf8adb4013954bd2972e1af9c0bff9e3, d185489eff86488f968d932466928c30; no voters:
I20260807 15:11:45.348541 5505 raft_consensus.cc:2781] T 1209f505a81349b3bfa86cffb317a288 P cf8adb4013954bd2972e1af9c0bff9e3 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:11:45.348680 5505 raft_consensus.cc:686] T 1209f505a81349b3bfa86cffb317a288 P cf8adb4013954bd2972e1af9c0bff9e3 [term 1 LEADER]: Becoming Leader. State: Replica: cf8adb4013954bd2972e1af9c0bff9e3, State: Running, Role: LEADER
I20260807 15:11:45.348762 5505 consensus_queue.cc:237] T 1209f505a81349b3bfa86cffb317a288 P cf8adb4013954bd2972e1af9c0bff9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dea3c7495b1649968e4c4f85f46fad98" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 40227 } } peers { permanent_uuid: "cf8adb4013954bd2972e1af9c0bff9e3" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 41777 } } peers { permanent_uuid: "d185489eff86488f968d932466928c30" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 36235 } }
I20260807 15:11:45.349646 4803 catalog_manager.cc:5495] T 1209f505a81349b3bfa86cffb317a288 P cf8adb4013954bd2972e1af9c0bff9e3 reported cstate change: term changed from 0 to 1, leader changed from <none> to cf8adb4013954bd2972e1af9c0bff9e3 (127.29.194.3). New cstate: current_term: 1 leader_uuid: "cf8adb4013954bd2972e1af9c0bff9e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dea3c7495b1649968e4c4f85f46fad98" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 40227 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cf8adb4013954bd2972e1af9c0bff9e3" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 41777 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d185489eff86488f968d932466928c30" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 36235 } health_report { overall_health: UNKNOWN } } }
I20260807 15:11:45.357936 5490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:45.356960 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54682 (local address 127.29.194.3:41777)
0807 15:11:45.357007 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:45.357009 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:45.357016 (+ 7us) server_negotiation.cc:407] Connection header received
0807 15:11:45.357051 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:45.357052 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:45.357066 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:45.357108 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:11:45.357748 (+ 640us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:11:45.357749 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:11:45.357749 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:11:45.357753 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:11:45.357769 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:11:45.357773 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:11:45.357821 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0807 15:11:45.357850 (+ 29us) server_negotiation.cc:299] Negotiation successful
0807 15:11:45.357862 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
W20260807 15:11:45.358137 5281 tablet.cc:2369] T 1209f505a81349b3bfa86cffb317a288 P d185489eff86488f968d932466928c30: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 15:11:45.359984 5214 raft_consensus.cc:1261] T 1209f505a81349b3bfa86cffb317a288 P d185489eff86488f968d932466928c30 [term 1 FOLLOWER]: Refusing update from remote peer cf8adb4013954bd2972e1af9c0bff9e3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:11:45.360040 5017 raft_consensus.cc:1261] T 1209f505a81349b3bfa86cffb317a288 P dea3c7495b1649968e4c4f85f46fad98 [term 1 FOLLOWER]: Refusing update from remote peer cf8adb4013954bd2972e1af9c0bff9e3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:11:45.360669 5509 consensus_queue.cc:1035] T 1209f505a81349b3bfa86cffb317a288 P cf8adb4013954bd2972e1af9c0bff9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dea3c7495b1649968e4c4f85f46fad98" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:11:45.360868 5505 consensus_queue.cc:1035] T 1209f505a81349b3bfa86cffb317a288 P cf8adb4013954bd2972e1af9c0bff9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d185489eff86488f968d932466928c30" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 36235 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:11:45.364512 5513 mvcc.cc:205] Tried to move back new op lower bound from 7315929109951205376 to 7315929109909155840. Current Snapshot: MvccSnapshot[applied={T|T < 7315929109951205376}]
I20260807 15:11:45.365223 5514 mvcc.cc:205] Tried to move back new op lower bound from 7315929109951205376 to 7315929109909155840. Current Snapshot: MvccSnapshot[applied={T|T < 7315929109951205376}]
I20260807 15:11:45.367296 4803 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:42248:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 15:11:45.367434 4803 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:42248:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 15:11:45.368322 4803 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 29c3551afcdf4c949e23f7e93bb8aa3c: Sending DeleteTablet for 3 replicas of tablet 1209f505a81349b3bfa86cffb317a288
I20260807 15:11:45.368738 5374 tablet_service.cc:1514] Processing DeleteTablet for tablet 1209f505a81349b3bfa86cffb317a288 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:11:45 UTC) from {username='slave'} at 127.0.0.1:54668
I20260807 15:11:45.368757 5187 tablet_service.cc:1514] Processing DeleteTablet for tablet 1209f505a81349b3bfa86cffb317a288 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:11:45 UTC) from {username='slave'} at 127.0.0.1:48312
I20260807 15:11:45.369019 30472 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7fZSIL/build/release/bin/kudu with pid 4895
I20260807 15:11:45.369160 4984 tablet_service.cc:1514] Processing DeleteTablet for tablet 1209f505a81349b3bfa86cffb317a288 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:11:45 UTC) from {username='slave'} at 127.0.0.1:60518
I20260807 15:11:45.373165 5525 tablet_replica.cc:330] T 1209f505a81349b3bfa86cffb317a288 P cf8adb4013954bd2972e1af9c0bff9e3: stopping tablet replica
I20260807 15:11:45.373431 5525 raft_consensus.cc:2229] T 1209f505a81349b3bfa86cffb317a288 P cf8adb4013954bd2972e1af9c0bff9e3 [term 1 LEADER]: Raft consensus shutting down.
I20260807 15:11:45.373634 5525 raft_consensus.cc:2258] T 1209f505a81349b3bfa86cffb317a288 P cf8adb4013954bd2972e1af9c0bff9e3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:11:45.373817 5523 tablet_replica.cc:330] T 1209f505a81349b3bfa86cffb317a288 P d185489eff86488f968d932466928c30: stopping tablet replica
I20260807 15:11:45.373971 5523 raft_consensus.cc:2229] T 1209f505a81349b3bfa86cffb317a288 P d185489eff86488f968d932466928c30 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 15:11:45.374092 5523 raft_consensus.cc:2258] T 1209f505a81349b3bfa86cffb317a288 P d185489eff86488f968d932466928c30 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:11:45.375203 5525 ts_tablet_manager.cc:1905] T 1209f505a81349b3bfa86cffb317a288 P cf8adb4013954bd2972e1af9c0bff9e3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:11:45.375491 5523 ts_tablet_manager.cc:1905] T 1209f505a81349b3bfa86cffb317a288 P d185489eff86488f968d932466928c30: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:11:45.377542 5523 ts_tablet_manager.cc:1918] T 1209f505a81349b3bfa86cffb317a288 P d185489eff86488f968d932466928c30: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 15:11:45.377576 5525 ts_tablet_manager.cc:1918] T 1209f505a81349b3bfa86cffb317a288 P cf8adb4013954bd2972e1af9c0bff9e3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 15:11:45.377898 5525 log.cc:1196] T 1209f505a81349b3bfa86cffb317a288 P cf8adb4013954bd2972e1af9c0bff9e3: Deleting WAL directory at /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115480851507-30472-0/minicluster-data/ts-2/wal/wals/1209f505a81349b3bfa86cffb317a288
I20260807 15:11:45.377893 5523 log.cc:1196] T 1209f505a81349b3bfa86cffb317a288 P d185489eff86488f968d932466928c30: Deleting WAL directory at /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115480851507-30472-0/minicluster-data/ts-1/wal/wals/1209f505a81349b3bfa86cffb317a288
I20260807 15:11:45.378407 5523 ts_tablet_manager.cc:1939] T 1209f505a81349b3bfa86cffb317a288 P d185489eff86488f968d932466928c30: Deleting consensus metadata
I20260807 15:11:45.378437 5525 ts_tablet_manager.cc:1939] T 1209f505a81349b3bfa86cffb317a288 P cf8adb4013954bd2972e1af9c0bff9e3: Deleting consensus metadata
I20260807 15:11:45.379125 4784 catalog_manager.cc:4841] TS d185489eff86488f968d932466928c30 (127.29.194.2:36235): tablet 1209f505a81349b3bfa86cffb317a288 (table test-table [id=7599929436e44a6ca9fe498a8dbc85eb]) successfully deleted
I20260807 15:11:45.379279 4784 catalog_manager.cc:4841] TS cf8adb4013954bd2972e1af9c0bff9e3 (127.29.194.3:41777): tablet 1209f505a81349b3bfa86cffb317a288 (table test-table [id=7599929436e44a6ca9fe498a8dbc85eb]) successfully deleted
W20260807 15:11:45.389802 4784 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.29.194.1:40227 (error 108)
I20260807 15:11:45.390153 30472 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7fZSIL/build/release/bin/kudu with pid 5088
I20260807 15:11:45.392987 5488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:45.390275 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.1:40227 (local address 127.0.0.1:60534)
0807 15:11:45.392861 (+ 2586us) negotiation.cc:107] Waiting for socket to connect
0807 15:11:45.392939 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.194.1:40227: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2555}
W20260807 15:11:45.398181 4784 catalog_manager.cc:4584] TS dea3c7495b1649968e4c4f85f46fad98 (127.29.194.1:40227): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 1209f505a81349b3bfa86cffb317a288: Network error: Client connection negotiation failed: client connection to 127.29.194.1:40227: connect: Connection refused (error 111)
I20260807 15:11:45.403623 30472 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7fZSIL/build/release/bin/kudu with pid 5284
I20260807 15:11:45.417385 30472 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7fZSIL/build/release/bin/kudu with pid 4761
2026-08-07T15:11:45Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (775 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260807 15:11:45.447175 30472 mini_oidc.cc:152] Starting JWKS server
I20260807 15:11:45.448382 30472 webserver.cc:460] Webserver started at https://127.0.0.1:36959/ using document root <none> and password file <none>
I20260807 15:11:45.448503 30472 mini_oidc.cc:189] Starting OIDC Discovery server
I20260807 15:11:45.448684 30472 webserver.cc:460] Webserver started at http://127.0.0.1:41195/ using document root <none> and password file <none>
2026-08-07T15:11:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T15:11:45Z Disabled control of system clock
I20260807 15:11:45.460219 30472 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7fZSIL/build/release/bin/kudu
/tmp/dist-test-task7fZSIL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786115480851507-30472-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786115480851507-30472-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786115480851507-30472-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786115480851507-30472-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.194.62:40795
--webserver_interface=127.29.194.62
--webserver_port=0
--builtin_ntp_servers=127.29.194.20:39385
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.194.62:40795
--enable_jwt_token_auth=true
--jwks_url=https://localhost:36959/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786115480851507-30472-0/testchainca.cert with env {}
W20260807 15:11:45.559180 5535 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:11:45.559487 5535 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:11:45.559515 5535 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:11:45.560882 5535 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:11:45.560942 5535 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:11:45.560953 5535 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:11:45.560966 5535 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:11:45.560976 5535 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260807 15:11:45.560988 5535 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260807 15:11:45.560998 5535 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:36959/jwks.json
I20260807 15:11:45.562937 5535 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.194.20:39385
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786115480851507-30472-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786115480851507-30472-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.194.62:40795
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.194.62:40795
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:36959/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786115480851507-30472-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.194.62
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786115480851507-30472-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5535
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786115480851507-30472-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:11:45.563222 5535 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:11:45.563853 5535 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:11:45.575032 5548 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:11:45.575032 5545 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:11:45.575057 5535 server_base.cc:1034] running on GCE node
W20260807 15:11:45.575032 5543 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:11:45.576155 5535 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:11:45.576589 5535 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:11:45.577757 5535 hybrid_clock.cc:648] HybridClock initialized: now 1786115505577746 us; error 35 us; skew 500 ppm
I20260807 15:11:45.579509 5535 webserver.cc:460] Webserver started at http://127.29.194.62:34979/ using document root <none> and password file <none>
I20260807 15:11:45.579928 5535 fs_manager.cc:359] Metadata directory not provided
I20260807 15:11:45.580006 5535 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:11:45.580163 5535 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:11:45.581676 5535 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786115480851507-30472-0/minicluster-data/master-0/data/instance:
uuid: "cdf0706673c5424badfaa3e7a01b4d6f"
format_stamp: "Formatted at 2026-08-07 15:11:45 on dist-test-slave-42z9"
server_key: "29d83ecbdaead5f5538c40e6e671ddfa"
server_key_iv: "5bd900f8722e95f70fbc52a11f0a9174"
server_key_version: "encryptionkey@0"
I20260807 15:11:45.582378 5535 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786115480851507-30472-0/minicluster-data/master-0/wal/instance:
uuid: "cdf0706673c5424badfaa3e7a01b4d6f"
format_stamp: "Formatted at 2026-08-07 15:11:45 on dist-test-slave-42z9"
server_key: "29d83ecbdaead5f5538c40e6e671ddfa"
server_key_iv: "5bd900f8722e95f70fbc52a11f0a9174"
server_key_version: "encryptionkey@0"
I20260807 15:11:45.584311 5535 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.001s
I20260807 15:11:45.585531 5559 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:11:45.585860 5535 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:11:45.585981 5535 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786115480851507-30472-0/minicluster-data/master-0/data,/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786115480851507-30472-0/minicluster-data/master-0/wal
uuid: "cdf0706673c5424badfaa3e7a01b4d6f"
format_stamp: "Formatted at 2026-08-07 15:11:45 on dist-test-slave-42z9"
server_key: "29d83ecbdaead5f5538c40e6e671ddfa"
server_key_iv: "5bd900f8722e95f70fbc52a11f0a9174"
server_key_version: "encryptionkey@0"
I20260807 15:11:45.586097 5535 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786115480851507-30472-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786115480851507-30472-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786115480851507-30472-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:11:45.609249 5535 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:11:45.609673 5535 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:11:45.609799 5535 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:11:45.614861 5535 rpc_server.cc:307] RPC server started. Bound to: 127.29.194.62:40795
I20260807 15:11:45.614907 5634 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.194.62:40795 every 8 connection(s)
I20260807 15:11:45.615448 5535 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786115480851507-30472-0/minicluster-data/master-0/data/info.pb
I20260807 15:11:45.616742 5635 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:11:45.617328 30472 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7fZSIL/build/release/bin/kudu as pid 5535
I20260807 15:11:45.617475 30472 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786115480851507-30472-0/minicluster-data/master-0/wal/instance
I20260807 15:11:45.617651 30472 external_mini_cluster.cc:1428] Setting key 03f214e1f0c0ffdf79a66acccc5bf7d0
I20260807 15:11:45.619736 5635 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cdf0706673c5424badfaa3e7a01b4d6f: Bootstrap starting.
I20260807 15:11:45.620379 5635 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cdf0706673c5424badfaa3e7a01b4d6f: Neither blocks nor log segments found. Creating new log.
I20260807 15:11:45.620685 5635 log.cc:824] T 00000000000000000000000000000000 P cdf0706673c5424badfaa3e7a01b4d6f: Log is configured to *not* fsync() on all Append() calls
I20260807 15:11:45.621784 5635 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cdf0706673c5424badfaa3e7a01b4d6f: No bootstrap required, opened a new log
I20260807 15:11:45.624446 5635 raft_consensus.cc:348] T 00000000000000000000000000000000 P cdf0706673c5424badfaa3e7a01b4d6f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cdf0706673c5424badfaa3e7a01b4d6f" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 40795 } }
I20260807 15:11:45.624626 5635 raft_consensus.cc:374] T 00000000000000000000000000000000 P cdf0706673c5424badfaa3e7a01b4d6f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:11:45.624662 5635 raft_consensus.cc:729] T 00000000000000000000000000000000 P cdf0706673c5424badfaa3e7a01b4d6f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cdf0706673c5424badfaa3e7a01b4d6f, State: Initialized, Role: FOLLOWER
I20260807 15:11:45.624796 5635 consensus_queue.cc:260] T 00000000000000000000000000000000 P cdf0706673c5424badfaa3e7a01b4d6f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cdf0706673c5424badfaa3e7a01b4d6f" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 40795 } }
I20260807 15:11:45.624881 5635 raft_consensus.cc:388] T 00000000000000000000000000000000 P cdf0706673c5424badfaa3e7a01b4d6f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:11:45.624922 5635 raft_consensus.cc:482] T 00000000000000000000000000000000 P cdf0706673c5424badfaa3e7a01b4d6f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:11:45.624970 5635 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cdf0706673c5424badfaa3e7a01b4d6f [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:11:45.625717 5635 raft_consensus.cc:504] T 00000000000000000000000000000000 P cdf0706673c5424badfaa3e7a01b4d6f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cdf0706673c5424badfaa3e7a01b4d6f" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 40795 } }
I20260807 15:11:45.625844 5635 leader_election.cc:302] T 00000000000000000000000000000000 P cdf0706673c5424badfaa3e7a01b4d6f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cdf0706673c5424badfaa3e7a01b4d6f; no voters:
I20260807 15:11:45.625977 5643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:45.618512 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50734 (local address 127.29.194.62:40795)
0807 15:11:45.618825 (+ 313us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:45.618836 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:45.618983 (+ 147us) server_negotiation.cc:407] Connection header received
0807 15:11:45.619783 (+ 800us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:45.619797 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:45.619905 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:45.620204 (+ 299us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:11:45.620514 (+ 310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:45.621851 (+ 1337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:45.622761 (+ 910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:45.623245 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:45.623368 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:45.624666 (+ 1298us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:11:45.624672 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:11:45.624673 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:11:45.625160 (+ 487us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:11:45.625242 (+ 82us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:11:45.625262 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:11:45.625349 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0807 15:11:45.625489 (+ 140us) server_negotiation.cc:299] Negotiation successful
0807 15:11:45.625626 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":209,"thread_start_us":110,"threads_started":1}
I20260807 15:11:45.626032 5635 leader_election.cc:288] T 00000000000000000000000000000000 P cdf0706673c5424badfaa3e7a01b4d6f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 15:11:45.626174 5647 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cdf0706673c5424badfaa3e7a01b4d6f [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:11:45.626319 5635 sys_catalog.cc:564] T 00000000000000000000000000000000 P cdf0706673c5424badfaa3e7a01b4d6f [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:11:45.626421 5647 raft_consensus.cc:686] T 00000000000000000000000000000000 P cdf0706673c5424badfaa3e7a01b4d6f [term 1 LEADER]: Becoming Leader. State: Replica: cdf0706673c5424badfaa3e7a01b4d6f, State: Running, Role: LEADER
I20260807 15:11:45.626528 5647 consensus_queue.cc:237] T 00000000000000000000000000000000 P cdf0706673c5424badfaa3e7a01b4d6f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cdf0706673c5424badfaa3e7a01b4d6f" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 40795 } }
I20260807 15:11:45.626922 5651 sys_catalog.cc:455] T 00000000000000000000000000000000 P cdf0706673c5424badfaa3e7a01b4d6f [sys.catalog]: SysCatalogTable state changed. Reason: New leader cdf0706673c5424badfaa3e7a01b4d6f. Latest consensus state: current_term: 1 leader_uuid: "cdf0706673c5424badfaa3e7a01b4d6f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cdf0706673c5424badfaa3e7a01b4d6f" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 40795 } } }
I20260807 15:11:45.626991 5648 sys_catalog.cc:455] T 00000000000000000000000000000000 P cdf0706673c5424badfaa3e7a01b4d6f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cdf0706673c5424badfaa3e7a01b4d6f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cdf0706673c5424badfaa3e7a01b4d6f" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 40795 } } }
I20260807 15:11:45.627151 30472 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 15:11:45.627118 5651 sys_catalog.cc:458] T 00000000000000000000000000000000 P cdf0706673c5424badfaa3e7a01b4d6f [sys.catalog]: This master's current role is: LEADER
I20260807 15:11:45.627852 5648 sys_catalog.cc:458] T 00000000000000000000000000000000 P cdf0706673c5424badfaa3e7a01b4d6f [sys.catalog]: This master's current role is: LEADER
W20260807 15:11:45.628294 5669 catalog_manager.cc:1558] T 00000000000000000000000000000000 P cdf0706673c5424badfaa3e7a01b4d6f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 15:11:45.628353 5669 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 15:11:45.629757 5671 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:11:45.630059 5671 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:11:45.631973 5671 catalog_manager.cc:1347] Generated new cluster ID: 432083a9a72842f59d5a0cd1c45fa078
I20260807 15:11:45.632055 5671 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:11:45.642143 5671 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:11:45.643066 5671 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:11:45.646008 5671 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cdf0706673c5424badfaa3e7a01b4d6f: Generated new TSK 0
I20260807 15:11:45.646216 5671 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260807 15:11:45.676400 5686 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260807 15:11:45.676538 5686 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
W20260807 15:11:45.677091 5685 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:11:45.660661 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:40795 (local address 127.0.0.1:50744)
0807 15:11:45.661076 (+ 415us) negotiation.cc:107] Waiting for socket to connect
0807 15:11:45.661090 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 15:11:45.661191 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:11:45.661401 (+ 210us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:11:45.661402 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:11:45.661403 (+ 1us) client_negotiation.cc:189] Negotiated authn=JWT
0807 15:11:45.661506 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:45.661509 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:45.662208 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:45.662209 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:45.662850 (+ 641us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:45.662852 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:45.663232 (+ 380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:45.663233 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:45.663299 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:45.663331 (+ 32us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0807 15:11:45.676642 (+ 13311us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36959/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0807 15:11:45.676744 (+ 102us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.29.194.62:40795: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36959/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":332,"thread_start_us":227,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.29.194.62:40795: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36959/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260807 15:11:45.677469 5643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:45.660822 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50744 (local address 127.29.194.62:40795)
0807 15:11:45.660916 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:45.660919 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:45.661179 (+ 260us) server_negotiation.cc:407] Connection header received
0807 15:11:45.661317 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:45.661318 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:45.661330 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:45.661390 (+ 60us) server_negotiation.cc:226] Negotiated authn=JWT
0807 15:11:45.661570 (+ 180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:45.662117 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:45.662946 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:45.663146 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:45.663204 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:45.663454 (+ 250us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0807 15:11:45.676465 (+ 13011us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36959/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0807 15:11:45.677352 (+ 887us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:50744: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36959/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":54}
I20260807 15:11:45.677476 30472 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7fZSIL/build/release/bin/kudu with pid 5535
W20260807 15:11:45.677546 5643 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:50744: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36959/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
2026-08-07T15:11:45Z chronyd exiting
I20260807 15:11:45.723656 30472 test_util.cc:175] -----------------------------------------------
I20260807 15:11:45.723770 30472 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786115480851507-30472-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (277 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260807 15:11:45.724736 30472 mini_oidc.cc:152] Starting JWKS server
I20260807 15:11:45.725482 30472 webserver.cc:460] Webserver started at https://127.0.0.1:44817/ using document root <none> and password file <none>
I20260807 15:11:45.725550 30472 mini_oidc.cc:189] Starting OIDC Discovery server
I20260807 15:11:45.725696 30472 webserver.cc:460] Webserver started at http://127.0.0.1:34919/ using document root <none> and password file <none>
2026-08-07T15:11:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T15:11:45Z Disabled control of system clock
I20260807 15:11:45.736590 30472 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7fZSIL/build/release/bin/kudu
/tmp/dist-test-task7fZSIL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786115480851507-30472-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786115480851507-30472-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786115480851507-30472-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786115480851507-30472-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.194.62:37183
--webserver_interface=127.29.194.62
--webserver_port=0
--builtin_ntp_servers=127.29.194.20:37023
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.194.62:37183
--enable_jwt_token_auth=true
--jwks_url=https://localhost:44817/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786115480851507-30472-0/testchainca.cert with env {}
W20260807 15:11:45.827710 5694 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:11:45.827998 5694 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:11:45.828065 5694 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:11:45.829763 5694 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:11:45.829851 5694 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:11:45.829869 5694 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:11:45.829885 5694 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:11:45.829900 5694 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260807 15:11:45.829916 5694 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260807 15:11:45.829931 5694 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:44817/jwks.json
I20260807 15:11:45.832185 5694 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.194.20:37023
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786115480851507-30472-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786115480851507-30472-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.194.62:37183
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.194.62:37183
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:44817/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786115480851507-30472-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.194.62
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786115480851507-30472-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5694
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786115480851507-30472-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:11:45.832460 5694 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:11:45.832645 5694 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:11:45.838279 5705 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:11:45.838234 5707 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:11:45.838241 5702 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:11:45.838353 5694 server_base.cc:1034] running on GCE node
I20260807 15:11:45.838910 5694 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:11:45.839354 5694 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:11:45.840553 5694 hybrid_clock.cc:648] HybridClock initialized: now 1786115505840520 us; error 54 us; skew 500 ppm
I20260807 15:11:45.842257 5694 webserver.cc:460] Webserver started at http://127.29.194.62:44703/ using document root <none> and password file <none>
I20260807 15:11:45.842519 5694 fs_manager.cc:359] Metadata directory not provided
I20260807 15:11:45.842578 5694 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:11:45.842712 5694 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:11:45.843850 5694 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786115480851507-30472-0/minicluster-data/master-0/data/instance:
uuid: "80fef75dbc4e4e30ae0069cf8b7734b2"
format_stamp: "Formatted at 2026-08-07 15:11:45 on dist-test-slave-42z9"
server_key: "7ca4fcd16c9f42606aee01df4bb4e2ba"
server_key_iv: "159ceddfc3a3e1c7e5ef3de1bc72654e"
server_key_version: "encryptionkey@0"
I20260807 15:11:45.844267 5694 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786115480851507-30472-0/minicluster-data/master-0/wal/instance:
uuid: "80fef75dbc4e4e30ae0069cf8b7734b2"
format_stamp: "Formatted at 2026-08-07 15:11:45 on dist-test-slave-42z9"
server_key: "7ca4fcd16c9f42606aee01df4bb4e2ba"
server_key_iv: "159ceddfc3a3e1c7e5ef3de1bc72654e"
server_key_version: "encryptionkey@0"
I20260807 15:11:45.845723 5694 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.001s
I20260807 15:11:45.846582 5716 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:11:45.846740 5694 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:11:45.846832 5694 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786115480851507-30472-0/minicluster-data/master-0/data,/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786115480851507-30472-0/minicluster-data/master-0/wal
uuid: "80fef75dbc4e4e30ae0069cf8b7734b2"
format_stamp: "Formatted at 2026-08-07 15:11:45 on dist-test-slave-42z9"
server_key: "7ca4fcd16c9f42606aee01df4bb4e2ba"
server_key_iv: "159ceddfc3a3e1c7e5ef3de1bc72654e"
server_key_version: "encryptionkey@0"
I20260807 15:11:45.846930 5694 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786115480851507-30472-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786115480851507-30472-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786115480851507-30472-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:11:45.862661 5694 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:11:45.863054 5694 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:11:45.863198 5694 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:11:45.867662 5694 rpc_server.cc:307] RPC server started. Bound to: 127.29.194.62:37183
I20260807 15:11:45.867733 5791 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.194.62:37183 every 8 connection(s)
I20260807 15:11:45.868093 5694 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786115480851507-30472-0/minicluster-data/master-0/data/info.pb
I20260807 15:11:45.868901 5792 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:11:45.872136 5792 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 80fef75dbc4e4e30ae0069cf8b7734b2: Bootstrap starting.
I20260807 15:11:45.872862 5792 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 80fef75dbc4e4e30ae0069cf8b7734b2: Neither blocks nor log segments found. Creating new log.
I20260807 15:11:45.872987 30472 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7fZSIL/build/release/bin/kudu as pid 5694
I20260807 15:11:45.873097 30472 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786115480851507-30472-0/minicluster-data/master-0/wal/instance
I20260807 15:11:45.873224 5792 log.cc:824] T 00000000000000000000000000000000 P 80fef75dbc4e4e30ae0069cf8b7734b2: Log is configured to *not* fsync() on all Append() calls
I20260807 15:11:45.873396 30472 external_mini_cluster.cc:1428] Setting key 568ed6fb46b5684a40c42bf5619ec890
I20260807 15:11:45.874301 5792 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 80fef75dbc4e4e30ae0069cf8b7734b2: No bootstrap required, opened a new log
I20260807 15:11:45.877039 5792 raft_consensus.cc:348] T 00000000000000000000000000000000 P 80fef75dbc4e4e30ae0069cf8b7734b2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "80fef75dbc4e4e30ae0069cf8b7734b2" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 37183 } }
I20260807 15:11:45.877264 5792 raft_consensus.cc:374] T 00000000000000000000000000000000 P 80fef75dbc4e4e30ae0069cf8b7734b2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:11:45.877310 5792 raft_consensus.cc:729] T 00000000000000000000000000000000 P 80fef75dbc4e4e30ae0069cf8b7734b2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80fef75dbc4e4e30ae0069cf8b7734b2, State: Initialized, Role: FOLLOWER
I20260807 15:11:45.877485 5792 consensus_queue.cc:260] T 00000000000000000000000000000000 P 80fef75dbc4e4e30ae0069cf8b7734b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "80fef75dbc4e4e30ae0069cf8b7734b2" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 37183 } }
I20260807 15:11:45.877561 5792 raft_consensus.cc:388] T 00000000000000000000000000000000 P 80fef75dbc4e4e30ae0069cf8b7734b2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:11:45.877588 5792 raft_consensus.cc:482] T 00000000000000000000000000000000 P 80fef75dbc4e4e30ae0069cf8b7734b2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:11:45.877635 5792 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 80fef75dbc4e4e30ae0069cf8b7734b2 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:11:45.878468 5792 raft_consensus.cc:504] T 00000000000000000000000000000000 P 80fef75dbc4e4e30ae0069cf8b7734b2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "80fef75dbc4e4e30ae0069cf8b7734b2" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 37183 } }
I20260807 15:11:45.878599 5792 leader_election.cc:302] T 00000000000000000000000000000000 P 80fef75dbc4e4e30ae0069cf8b7734b2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 80fef75dbc4e4e30ae0069cf8b7734b2; no voters:
I20260807 15:11:45.878788 5792 leader_election.cc:288] T 00000000000000000000000000000000 P 80fef75dbc4e4e30ae0069cf8b7734b2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 15:11:45.878829 5804 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 80fef75dbc4e4e30ae0069cf8b7734b2 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:11:45.878938 5804 raft_consensus.cc:686] T 00000000000000000000000000000000 P 80fef75dbc4e4e30ae0069cf8b7734b2 [term 1 LEADER]: Becoming Leader. State: Replica: 80fef75dbc4e4e30ae0069cf8b7734b2, State: Running, Role: LEADER
I20260807 15:11:45.879040 5804 consensus_queue.cc:237] T 00000000000000000000000000000000 P 80fef75dbc4e4e30ae0069cf8b7734b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "80fef75dbc4e4e30ae0069cf8b7734b2" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 37183 } }
I20260807 15:11:45.879091 5792 sys_catalog.cc:564] T 00000000000000000000000000000000 P 80fef75dbc4e4e30ae0069cf8b7734b2 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:11:45.879410 5804 sys_catalog.cc:455] T 00000000000000000000000000000000 P 80fef75dbc4e4e30ae0069cf8b7734b2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 80fef75dbc4e4e30ae0069cf8b7734b2. Latest consensus state: current_term: 1 leader_uuid: "80fef75dbc4e4e30ae0069cf8b7734b2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "80fef75dbc4e4e30ae0069cf8b7734b2" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 37183 } } }
I20260807 15:11:45.879448 5805 sys_catalog.cc:455] T 00000000000000000000000000000000 P 80fef75dbc4e4e30ae0069cf8b7734b2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "80fef75dbc4e4e30ae0069cf8b7734b2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "80fef75dbc4e4e30ae0069cf8b7734b2" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 37183 } } }
I20260807 15:11:45.879684 5805 sys_catalog.cc:458] T 00000000000000000000000000000000 P 80fef75dbc4e4e30ae0069cf8b7734b2 [sys.catalog]: This master's current role is: LEADER
I20260807 15:11:45.879777 5803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:45.874028 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52010 (local address 127.29.194.62:37183)
0807 15:11:45.874246 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:45.874253 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:45.874279 (+ 26us) server_negotiation.cc:407] Connection header received
0807 15:11:45.874826 (+ 547us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:45.874834 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:45.874898 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:45.875026 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:11:45.875281 (+ 255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:45.876160 (+ 879us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:45.877295 (+ 1135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:45.877900 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:45.878007 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:45.878731 (+ 724us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:11:45.878736 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:11:45.878736 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:11:45.878957 (+ 221us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:11:45.879010 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:11:45.879051 (+ 41us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:11:45.879353 (+ 302us) server_negotiation.cc:1035] Waiting for connection context
0807 15:11:45.879392 (+ 39us) server_negotiation.cc:299] Negotiation successful
0807 15:11:45.879488 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137,"thread_start_us":87,"threads_started":1}
I20260807 15:11:45.879487 5804 sys_catalog.cc:458] T 00000000000000000000000000000000 P 80fef75dbc4e4e30ae0069cf8b7734b2 [sys.catalog]: This master's current role is: LEADER
I20260807 15:11:45.880504 5822 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:11:45.881078 30472 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 15:11:45.881204 5822 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:11:45.882943 5822 catalog_manager.cc:1347] Generated new cluster ID: 09194c0608754ee682040b564db06eb3
I20260807 15:11:45.883028 5822 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:11:45.888383 5822 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:11:45.889133 5822 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:11:45.896103 5822 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 80fef75dbc4e4e30ae0069cf8b7734b2: Generated new TSK 0
I20260807 15:11:45.896337 5822 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260807 15:11:45.929271 5843 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260807 15:11:45.929382 5843 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260807 15:11:45.929709 5803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:45.905313 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52022 (local address 127.29.194.62:37183)
0807 15:11:45.905494 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:45.905497 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:45.905517 (+ 20us) server_negotiation.cc:407] Connection header received
0807 15:11:45.905584 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:45.905586 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:45.905605 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:45.905741 (+ 136us) server_negotiation.cc:226] Negotiated authn=JWT
0807 15:11:45.906008 (+ 267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:45.906589 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:45.907797 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:45.908033 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:45.908097 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:45.908386 (+ 289us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0807 15:11:45.929427 (+ 21041us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44817/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0807 15:11:45.929638 (+ 211us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:52022: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44817/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":45}
W20260807 15:11:45.929768 5803 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:52022: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44817/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260807 15:11:45.929963 5842 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:11:45.905163 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:37183 (local address 127.0.0.1:52022)
0807 15:11:45.905348 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0807 15:11:45.905360 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 15:11:45.905416 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:11:45.905759 (+ 343us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:11:45.905760 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:11:45.905763 (+ 3us) client_negotiation.cc:189] Negotiated authn=JWT
0807 15:11:45.905842 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:45.905844 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:45.906693 (+ 849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:45.906694 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:45.907681 (+ 987us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:45.907684 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:45.908111 (+ 427us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:45.908111 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:45.908176 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:45.908184 (+ 8us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0807 15:11:45.929587 (+ 21403us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44817/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0807 15:11:45.929721 (+ 134us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.29.194.62:37183: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44817/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":100,"threads_started":1}
I20260807 15:11:45.930738 30472 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7fZSIL/build/release/bin/kudu with pid 5694
2026-08-07T15:11:45Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (252 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260807 15:11:45.976759 30472 mini_oidc.cc:152] Starting JWKS server
I20260807 15:11:45.977520 30472 webserver.cc:460] Webserver started at https://127.0.0.1:36181/ using document root <none> and password file <none>
I20260807 15:11:45.977571 30472 mini_oidc.cc:189] Starting OIDC Discovery server
I20260807 15:11:45.977694 30472 webserver.cc:460] Webserver started at http://127.0.0.1:34235/ using document root <none> and password file <none>
2026-08-07T15:11:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T15:11:45Z Disabled control of system clock
I20260807 15:11:45.988727 30472 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7fZSIL/build/release/bin/kudu
/tmp/dist-test-task7fZSIL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786115480851507-30472-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786115480851507-30472-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786115480851507-30472-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786115480851507-30472-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.194.62:42133
--webserver_interface=127.29.194.62
--webserver_port=0
--builtin_ntp_servers=127.29.194.20:38833
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.194.62:42133
--enable_jwt_token_auth=true
--jwks_url=https://localhost:36181/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260807 15:11:46.086215 5849 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:11:46.086496 5849 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:11:46.086522 5849 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:11:46.087894 5849 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:11:46.087947 5849 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:11:46.087958 5849 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:11:46.087970 5849 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:11:46.087981 5849 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260807 15:11:46.087994 5849 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260807 15:11:46.088004 5849 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:36181/jwks.json
I20260807 15:11:46.089618 5849 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.194.20:38833
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786115480851507-30472-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786115480851507-30472-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.194.62:42133
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.194.62:42133
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:36181/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786115480851507-30472-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.194.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5849
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786115480851507-30472-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:11:46.089887 5849 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:11:46.090180 5849 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:11:46.095969 5857 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:11:46.095969 5854 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:11:46.095988 5855 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:11:46.098105 5849 server_base.cc:1034] running on GCE node
I20260807 15:11:46.098438 5849 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:11:46.098770 5849 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:11:46.099953 5849 hybrid_clock.cc:648] HybridClock initialized: now 1786115506099927 us; error 48 us; skew 500 ppm
I20260807 15:11:46.101972 5849 webserver.cc:460] Webserver started at http://127.29.194.62:45395/ using document root <none> and password file <none>
I20260807 15:11:46.102595 5849 fs_manager.cc:359] Metadata directory not provided
I20260807 15:11:46.102696 5849 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:11:46.103135 5849 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:11:46.104573 5849 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786115480851507-30472-0/minicluster-data/master-0/data/instance:
uuid: "ee9a045e6c2048b5b3a32d0975a5c929"
format_stamp: "Formatted at 2026-08-07 15:11:46 on dist-test-slave-42z9"
server_key: "3f705451a0605daa2605ad59943439c9"
server_key_iv: "23a9a324bced608f33ab506a9ef7f0b9"
server_key_version: "encryptionkey@0"
I20260807 15:11:46.104972 5849 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786115480851507-30472-0/minicluster-data/master-0/wal/instance:
uuid: "ee9a045e6c2048b5b3a32d0975a5c929"
format_stamp: "Formatted at 2026-08-07 15:11:46 on dist-test-slave-42z9"
server_key: "3f705451a0605daa2605ad59943439c9"
server_key_iv: "23a9a324bced608f33ab506a9ef7f0b9"
server_key_version: "encryptionkey@0"
I20260807 15:11:46.107091 5849 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.005s sys 0.000s
I20260807 15:11:46.108161 5866 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:11:46.108460 5849 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:11:46.108588 5849 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786115480851507-30472-0/minicluster-data/master-0/data,/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786115480851507-30472-0/minicluster-data/master-0/wal
uuid: "ee9a045e6c2048b5b3a32d0975a5c929"
format_stamp: "Formatted at 2026-08-07 15:11:46 on dist-test-slave-42z9"
server_key: "3f705451a0605daa2605ad59943439c9"
server_key_iv: "23a9a324bced608f33ab506a9ef7f0b9"
server_key_version: "encryptionkey@0"
I20260807 15:11:46.108692 5849 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786115480851507-30472-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786115480851507-30472-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786115480851507-30472-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:11:46.125653 5849 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:11:46.126003 5849 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:11:46.126121 5849 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:11:46.130237 5849 rpc_server.cc:307] RPC server started. Bound to: 127.29.194.62:42133
I20260807 15:11:46.130280 5938 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.194.62:42133 every 8 connection(s)
I20260807 15:11:46.130635 5849 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786115480851507-30472-0/minicluster-data/master-0/data/info.pb
I20260807 15:11:46.131405 5939 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:11:46.134433 5939 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ee9a045e6c2048b5b3a32d0975a5c929: Bootstrap starting.
I20260807 15:11:46.135169 5939 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ee9a045e6c2048b5b3a32d0975a5c929: Neither blocks nor log segments found. Creating new log.
I20260807 15:11:46.135176 30472 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7fZSIL/build/release/bin/kudu as pid 5849
I20260807 15:11:46.135250 30472 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786115480851507-30472-0/minicluster-data/master-0/wal/instance
I20260807 15:11:46.135470 5939 log.cc:824] T 00000000000000000000000000000000 P ee9a045e6c2048b5b3a32d0975a5c929: Log is configured to *not* fsync() on all Append() calls
I20260807 15:11:46.135581 30472 external_mini_cluster.cc:1428] Setting key 155a7e7b8a4a77800c2f8773be1e13e3
I20260807 15:11:46.136446 5939 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ee9a045e6c2048b5b3a32d0975a5c929: No bootstrap required, opened a new log
I20260807 15:11:46.139545 5939 raft_consensus.cc:348] T 00000000000000000000000000000000 P ee9a045e6c2048b5b3a32d0975a5c929 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ee9a045e6c2048b5b3a32d0975a5c929" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 42133 } }
I20260807 15:11:46.139822 5939 raft_consensus.cc:374] T 00000000000000000000000000000000 P ee9a045e6c2048b5b3a32d0975a5c929 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:11:46.139863 5939 raft_consensus.cc:729] T 00000000000000000000000000000000 P ee9a045e6c2048b5b3a32d0975a5c929 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee9a045e6c2048b5b3a32d0975a5c929, State: Initialized, Role: FOLLOWER
I20260807 15:11:46.140027 5939 consensus_queue.cc:260] T 00000000000000000000000000000000 P ee9a045e6c2048b5b3a32d0975a5c929 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ee9a045e6c2048b5b3a32d0975a5c929" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 42133 } }
I20260807 15:11:46.140136 5939 raft_consensus.cc:388] T 00000000000000000000000000000000 P ee9a045e6c2048b5b3a32d0975a5c929 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:11:46.140172 5939 raft_consensus.cc:482] T 00000000000000000000000000000000 P ee9a045e6c2048b5b3a32d0975a5c929 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:11:46.140229 5939 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ee9a045e6c2048b5b3a32d0975a5c929 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:11:46.141676 5939 raft_consensus.cc:504] T 00000000000000000000000000000000 P ee9a045e6c2048b5b3a32d0975a5c929 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ee9a045e6c2048b5b3a32d0975a5c929" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 42133 } }
I20260807 15:11:46.141829 5939 leader_election.cc:302] T 00000000000000000000000000000000 P ee9a045e6c2048b5b3a32d0975a5c929 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ee9a045e6c2048b5b3a32d0975a5c929; no voters:
I20260807 15:11:46.142104 5939 leader_election.cc:288] T 00000000000000000000000000000000 P ee9a045e6c2048b5b3a32d0975a5c929 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 15:11:46.142256 5946 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ee9a045e6c2048b5b3a32d0975a5c929 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:11:46.142585 5939 sys_catalog.cc:564] T 00000000000000000000000000000000 P ee9a045e6c2048b5b3a32d0975a5c929 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:11:46.142767 5945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:46.136246 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59022 (local address 127.29.194.62:42133)
0807 15:11:46.136493 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:46.136503 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:46.136542 (+ 39us) server_negotiation.cc:407] Connection header received
0807 15:11:46.137099 (+ 557us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:46.137106 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:46.137169 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:46.137301 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:11:46.137565 (+ 264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:46.138529 (+ 964us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:46.139324 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:46.139935 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:46.140079 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:46.141414 (+ 1335us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:11:46.141421 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:11:46.141421 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:11:46.141696 (+ 275us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:11:46.141753 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:11:46.141847 (+ 94us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:11:46.142040 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0807 15:11:46.142207 (+ 167us) server_negotiation.cc:299] Negotiation successful
0807 15:11:46.142474 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":160,"thread_start_us":94,"threads_started":1}
I20260807 15:11:46.142778 5946 raft_consensus.cc:686] T 00000000000000000000000000000000 P ee9a045e6c2048b5b3a32d0975a5c929 [term 1 LEADER]: Becoming Leader. State: Replica: ee9a045e6c2048b5b3a32d0975a5c929, State: Running, Role: LEADER
I20260807 15:11:46.143009 5946 consensus_queue.cc:237] T 00000000000000000000000000000000 P ee9a045e6c2048b5b3a32d0975a5c929 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ee9a045e6c2048b5b3a32d0975a5c929" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 42133 } }
I20260807 15:11:46.143452 5946 sys_catalog.cc:455] T 00000000000000000000000000000000 P ee9a045e6c2048b5b3a32d0975a5c929 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ee9a045e6c2048b5b3a32d0975a5c929. Latest consensus state: current_term: 1 leader_uuid: "ee9a045e6c2048b5b3a32d0975a5c929" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ee9a045e6c2048b5b3a32d0975a5c929" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 42133 } } }
I20260807 15:11:46.143536 5946 sys_catalog.cc:458] T 00000000000000000000000000000000 P ee9a045e6c2048b5b3a32d0975a5c929 [sys.catalog]: This master's current role is: LEADER
I20260807 15:11:46.143775 5951 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:11:46.144351 5951 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:11:46.144500 5948 sys_catalog.cc:455] T 00000000000000000000000000000000 P ee9a045e6c2048b5b3a32d0975a5c929 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ee9a045e6c2048b5b3a32d0975a5c929" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ee9a045e6c2048b5b3a32d0975a5c929" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 42133 } } }
I20260807 15:11:46.144640 5948 sys_catalog.cc:458] T 00000000000000000000000000000000 P ee9a045e6c2048b5b3a32d0975a5c929 [sys.catalog]: This master's current role is: LEADER
I20260807 15:11:46.144695 30472 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 15:11:46.146559 5951 catalog_manager.cc:1347] Generated new cluster ID: e3c6dd4f924e423e96f3f201eab68f23
I20260807 15:11:46.146618 5951 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:11:46.152784 5951 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:11:46.153322 5951 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:11:46.159507 5951 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ee9a045e6c2048b5b3a32d0975a5c929: Generated new TSK 0
I20260807 15:11:46.159879 5951 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 15:11:46.173108 5945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:46.169561 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59028 (local address 127.29.194.62:42133)
0807 15:11:46.169620 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:46.169622 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:46.169724 (+ 102us) server_negotiation.cc:407] Connection header received
0807 15:11:46.169809 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:46.169810 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:46.169825 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:46.169889 (+ 64us) server_negotiation.cc:226] Negotiated authn=JWT
0807 15:11:46.170047 (+ 158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:46.170573 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:46.171393 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:46.171599 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:46.171657 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:46.171846 (+ 189us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0807 15:11:46.172841 (+ 995us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36181/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0807 15:11:46.173009 (+ 168us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:59028: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36181/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":31}
W20260807 15:11:46.173183 5945 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:59028: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36181/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260807 15:11:46.173223 5982 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:11:46.169493 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:42133 (local address 127.0.0.1:59028)
0807 15:11:46.169652 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0807 15:11:46.169664 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 15:11:46.169733 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:11:46.169895 (+ 162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:11:46.169896 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:11:46.169898 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0807 15:11:46.169960 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:46.169963 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:46.170680 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:46.170682 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:46.171303 (+ 621us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:46.171305 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:46.171675 (+ 370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:46.171676 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:46.171738 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:46.171745 (+ 7us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0807 15:11:46.172966 (+ 1221us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36181/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0807 15:11:46.173105 (+ 139us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.29.194.62:42133: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36181/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":95,"threads_started":1}
I20260807 15:11:46.173821 30472 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7fZSIL/build/release/bin/kudu with pid 5849
E20260807 15:11:46.187575 5983 webserver.cc:533] Webserver: sslize failed:
E20260807 15:11:46.187731 5983 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-08-07T15:11:46Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (241 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-07T15:11:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T15:11:46Z Disabled control of system clock
I20260807 15:11:46.227406 30472 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7fZSIL/build/release/bin/kudu
/tmp/dist-test-task7fZSIL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786115480851507-30472-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786115480851507-30472-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786115480851507-30472-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786115480851507-30472-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.194.62:34617
--webserver_interface=127.29.194.62
--webserver_port=0
--builtin_ntp_servers=127.29.194.20:42927
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.194.62:34617
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786115480851507-30472-0/testchainca.cert with env {}
W20260807 15:11:46.320071 5987 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:11:46.320323 5987 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:11:46.320348 5987 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:11:46.321688 5987 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:11:46.321751 5987 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:11:46.321763 5987 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:11:46.321775 5987 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:11:46.321787 5987 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260807 15:11:46.321799 5987 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260807 15:11:46.321810 5987 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260807 15:11:46.323706 5987 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.194.20:42927
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786115480851507-30472-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786115480851507-30472-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.194.62:34617
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.194.62:34617
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786115480851507-30472-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.194.62
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786115480851507-30472-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5987
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786115480851507-30472-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:11:46.324020 5987 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:11:46.324262 5987 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:11:46.329667 5997 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:11:46.329700 5993 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:11:46.329730 5994 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:11:46.330149 5987 server_base.cc:1034] running on GCE node
I20260807 15:11:46.330420 5987 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:11:46.330721 5987 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:11:46.331902 5987 hybrid_clock.cc:648] HybridClock initialized: now 1786115506331881 us; error 39 us; skew 500 ppm
I20260807 15:11:46.333767 5987 webserver.cc:460] Webserver started at http://127.29.194.62:46437/ using document root <none> and password file <none>
I20260807 15:11:46.334038 5987 fs_manager.cc:359] Metadata directory not provided
I20260807 15:11:46.334093 5987 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:11:46.334227 5987 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:11:46.335367 5987 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786115480851507-30472-0/minicluster-data/master-0/data/instance:
uuid: "c6f8f7d85d3245519cae9529a7ab84ec"
format_stamp: "Formatted at 2026-08-07 15:11:46 on dist-test-slave-42z9"
server_key: "c101b54308237d06d6dd9e8aa2d10a57"
server_key_iv: "b60ad40418b702311f6cf062655dc662"
server_key_version: "encryptionkey@0"
I20260807 15:11:46.335733 5987 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786115480851507-30472-0/minicluster-data/master-0/wal/instance:
uuid: "c6f8f7d85d3245519cae9529a7ab84ec"
format_stamp: "Formatted at 2026-08-07 15:11:46 on dist-test-slave-42z9"
server_key: "c101b54308237d06d6dd9e8aa2d10a57"
server_key_iv: "b60ad40418b702311f6cf062655dc662"
server_key_version: "encryptionkey@0"
I20260807 15:11:46.337146 5987 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 15:11:46.337977 6008 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:11:46.338356 5987 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260807 15:11:46.338469 5987 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786115480851507-30472-0/minicluster-data/master-0/data,/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786115480851507-30472-0/minicluster-data/master-0/wal
uuid: "c6f8f7d85d3245519cae9529a7ab84ec"
format_stamp: "Formatted at 2026-08-07 15:11:46 on dist-test-slave-42z9"
server_key: "c101b54308237d06d6dd9e8aa2d10a57"
server_key_iv: "b60ad40418b702311f6cf062655dc662"
server_key_version: "encryptionkey@0"
I20260807 15:11:46.338541 5987 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786115480851507-30472-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786115480851507-30472-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786115480851507-30472-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:11:46.352528 5987 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:11:46.353118 5987 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:11:46.353302 5987 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:11:46.357784 5987 rpc_server.cc:307] RPC server started. Bound to: 127.29.194.62:34617
I20260807 15:11:46.357859 6088 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.194.62:34617 every 8 connection(s)
I20260807 15:11:46.358230 5987 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786115480851507-30472-0/minicluster-data/master-0/data/info.pb
I20260807 15:11:46.359373 6090 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:11:46.362031 6090 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c6f8f7d85d3245519cae9529a7ab84ec: Bootstrap starting.
I20260807 15:11:46.362655 6090 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c6f8f7d85d3245519cae9529a7ab84ec: Neither blocks nor log segments found. Creating new log.
I20260807 15:11:46.362910 6090 log.cc:824] T 00000000000000000000000000000000 P c6f8f7d85d3245519cae9529a7ab84ec: Log is configured to *not* fsync() on all Append() calls
I20260807 15:11:46.363627 6090 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c6f8f7d85d3245519cae9529a7ab84ec: No bootstrap required, opened a new log
I20260807 15:11:46.364971 30472 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7fZSIL/build/release/bin/kudu as pid 5987
I20260807 15:11:46.365059 30472 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786115480851507-30472-0/minicluster-data/master-0/wal/instance
I20260807 15:11:46.365325 30472 external_mini_cluster.cc:1428] Setting key eb2b9f692209572cfcf7b4a088fb207d
I20260807 15:11:46.365381 6090 raft_consensus.cc:348] T 00000000000000000000000000000000 P c6f8f7d85d3245519cae9529a7ab84ec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c6f8f7d85d3245519cae9529a7ab84ec" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 34617 } }
I20260807 15:11:46.365614 6090 raft_consensus.cc:374] T 00000000000000000000000000000000 P c6f8f7d85d3245519cae9529a7ab84ec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:11:46.365664 6090 raft_consensus.cc:729] T 00000000000000000000000000000000 P c6f8f7d85d3245519cae9529a7ab84ec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c6f8f7d85d3245519cae9529a7ab84ec, State: Initialized, Role: FOLLOWER
I20260807 15:11:46.365888 6090 consensus_queue.cc:260] T 00000000000000000000000000000000 P c6f8f7d85d3245519cae9529a7ab84ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c6f8f7d85d3245519cae9529a7ab84ec" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 34617 } }
I20260807 15:11:46.365979 6090 raft_consensus.cc:388] T 00000000000000000000000000000000 P c6f8f7d85d3245519cae9529a7ab84ec [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:11:46.366014 6090 raft_consensus.cc:482] T 00000000000000000000000000000000 P c6f8f7d85d3245519cae9529a7ab84ec [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:11:46.366060 6090 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c6f8f7d85d3245519cae9529a7ab84ec [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:11:46.366719 6090 raft_consensus.cc:504] T 00000000000000000000000000000000 P c6f8f7d85d3245519cae9529a7ab84ec [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c6f8f7d85d3245519cae9529a7ab84ec" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 34617 } }
I20260807 15:11:46.366838 6090 leader_election.cc:302] T 00000000000000000000000000000000 P c6f8f7d85d3245519cae9529a7ab84ec [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c6f8f7d85d3245519cae9529a7ab84ec; no voters:
I20260807 15:11:46.366990 6090 leader_election.cc:288] T 00000000000000000000000000000000 P c6f8f7d85d3245519cae9529a7ab84ec [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 15:11:46.367180 6096 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c6f8f7d85d3245519cae9529a7ab84ec [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:11:46.367273 6090 sys_catalog.cc:564] T 00000000000000000000000000000000 P c6f8f7d85d3245519cae9529a7ab84ec [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:11:46.367352 6096 raft_consensus.cc:686] T 00000000000000000000000000000000 P c6f8f7d85d3245519cae9529a7ab84ec [term 1 LEADER]: Becoming Leader. State: Replica: c6f8f7d85d3245519cae9529a7ab84ec, State: Running, Role: LEADER
I20260807 15:11:46.367450 6096 consensus_queue.cc:237] T 00000000000000000000000000000000 P c6f8f7d85d3245519cae9529a7ab84ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c6f8f7d85d3245519cae9529a7ab84ec" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 34617 } }
I20260807 15:11:46.367862 6098 sys_catalog.cc:455] T 00000000000000000000000000000000 P c6f8f7d85d3245519cae9529a7ab84ec [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c6f8f7d85d3245519cae9529a7ab84ec" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c6f8f7d85d3245519cae9529a7ab84ec" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 34617 } } }
I20260807 15:11:46.367959 6098 sys_catalog.cc:458] T 00000000000000000000000000000000 P c6f8f7d85d3245519cae9529a7ab84ec [sys.catalog]: This master's current role is: LEADER
I20260807 15:11:46.368153 6096 sys_catalog.cc:455] T 00000000000000000000000000000000 P c6f8f7d85d3245519cae9529a7ab84ec [sys.catalog]: SysCatalogTable state changed. Reason: New leader c6f8f7d85d3245519cae9529a7ab84ec. Latest consensus state: current_term: 1 leader_uuid: "c6f8f7d85d3245519cae9529a7ab84ec" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c6f8f7d85d3245519cae9529a7ab84ec" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 34617 } } }
I20260807 15:11:46.368213 6096 sys_catalog.cc:458] T 00000000000000000000000000000000 P c6f8f7d85d3245519cae9529a7ab84ec [sys.catalog]: This master's current role is: LEADER
I20260807 15:11:46.368595 6116 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:11:46.369381 6116 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:11:46.371420 6094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:46.365993 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47868 (local address 127.29.194.62:34617)
0807 15:11:46.366232 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:46.366240 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:46.366265 (+ 25us) server_negotiation.cc:407] Connection header received
0807 15:11:46.366391 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:46.366396 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:46.366459 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:46.366564 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:11:46.366881 (+ 317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:46.368282 (+ 1401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:46.369319 (+ 1037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:46.369810 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:46.369933 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:46.370723 (+ 790us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:11:46.370726 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:11:46.370727 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:11:46.370903 (+ 176us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:11:46.370937 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:11:46.370948 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:11:46.371005 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0807 15:11:46.371117 (+ 112us) server_negotiation.cc:299] Negotiation successful
0807 15:11:46.371204 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":163,"thread_start_us":101,"threads_started":1}
I20260807 15:11:46.371840 6116 catalog_manager.cc:1347] Generated new cluster ID: e474ac705a394ed38d2715c116cd4201
I20260807 15:11:46.371906 6116 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:11:46.372455 30472 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 15:11:46.389592 6116 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:11:46.390182 6116 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:11:46.393371 6116 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c6f8f7d85d3245519cae9529a7ab84ec: Generated new TSK 0
I20260807 15:11:46.393635 6116 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 15:11:47.414876 6094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:46.406056 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47880 (local address 127.29.194.62:34617)
0807 15:11:46.406111 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:46.406113 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:46.406217 (+ 104us) server_negotiation.cc:407] Connection header received
0807 15:11:46.406308 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:46.406309 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:46.406320 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:46.406388 (+ 68us) server_negotiation.cc:226] Negotiated authn=JWT
0807 15:11:46.406523 (+ 135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:46.407074 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:46.407853 (+ 779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:46.408091 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:46.408150 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:46.408346 (+ 196us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0807 15:11:47.414568 (+1006222us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0807 15:11:47.414792 (+ 224us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:47880: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":28}
W20260807 15:11:47.414965 6144 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:11:46.405973 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:34617 (local address 127.0.0.1:47880)
0807 15:11:46.406142 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0807 15:11:46.406153 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 15:11:46.406226 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:11:46.406391 (+ 165us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:11:46.406392 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:11:46.406394 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0807 15:11:46.406459 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:46.406461 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:46.407164 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:46.407165 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:46.407765 (+ 600us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:46.407767 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:46.408168 (+ 401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:46.408169 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:46.408228 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:46.408234 (+ 6us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0807 15:11:47.414747 (+1006513us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0807 15:11:47.414875 (+ 128us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.29.194.62:34617: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":100,"threads_started":1}
W20260807 15:11:47.414975 6094 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:47880: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
I20260807 15:11:47.415441 30472 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7fZSIL/build/release/bin/kudu with pid 5987
2026-08-07T15:11:47Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (1224 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (94 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (100 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115480851507-30472-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 15:11:47 dist-test-slave-42z9 krb5kdc[6163](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 15:11:47 dist-test-slave-42z9 krb5kdc[6163](info): set up 2 sockets
Aug 07 15:11:47 dist-test-slave-42z9 krb5kdc[6163](info): commencing operation
krb5kdc: starting...
W20260807 15:11:48.791594 30472 mini_kdc.cc:121] Time spent starting KDC: real 1.152s user 0.004s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115480851507-30472-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115480851507-30472-0/krb5kdc/test-user.keytab.
Aug 07 15:11:48 dist-test-slave-42z9 krb5kdc[6163](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115508, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T15:11:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T15:11:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.194.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.194.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.194.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.62.keytab.
Entry for principal kudu/127.29.194.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.62.keytab.
I20260807 15:11:48.934527 30472 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7fZSIL/build/release/bin/kudu
/tmp/dist-test-task7fZSIL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115480851507-30472-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115480851507-30472-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115480851507-30472-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115480851507-30472-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.194.62:38089
--webserver_interface=127.29.194.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.62.keytab
--principal=kudu/127.29.194.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.194.20:43329
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.194.62:38089
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115480851507-30472-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115480851507-30472-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115480851507-30472-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:11:49.030236 6181 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:11:49.030500 6181 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:11:49.030529 6181 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:11:49.031958 6181 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:11:49.032012 6181 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:11:49.032027 6181 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:11:49.032039 6181 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:11:49.032050 6181 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:11:49.033792 6181 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.194.20:43329
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115480851507-30472-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115480851507-30472-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.194.62:38089
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.62.keytab
--principal=kudu/127.29.194.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.194.62:38089
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115480851507-30472-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.194.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6181
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115480851507-30472-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:11:49.034085 6181 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:11:49.034309 6181 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 15:11:49.039949 6181 server_base.cc:1034] running on GCE node
W20260807 15:11:49.039868 6188 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:11:49.039996 6187 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:11:49.039868 6190 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:11:49.040396 6181 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:11:49.040685 6181 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:11:49.041846 6181 hybrid_clock.cc:648] HybridClock initialized: now 1786115509041823 us; error 33 us; skew 500 ppm
Aug 07 15:11:49 dist-test-slave-42z9 krb5kdc[6163](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115509, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.194.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:11:49.048584 6181 init.cc:377] Logged in from keytab as kudu/127.29.194.62@KRBTEST.COM (short username kudu)
I20260807 15:11:49.049095 6181 webserver.cc:460] Webserver started at http://127.29.194.62:38021/ using document root <none> and password file <none>
I20260807 15:11:49.049319 6181 fs_manager.cc:359] Metadata directory not provided
I20260807 15:11:49.049368 6181 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:11:49.049536 6181 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:11:49.050830 6181 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115480851507-30472-0/minicluster-data/master-0/data/instance:
uuid: "d526fffd10bd4d419a05f52b02285e6e"
format_stamp: "Formatted at 2026-08-07 15:11:49 on dist-test-slave-42z9"
server_key: "8b683a58ef3b2811b60d42932a919b76"
server_key_iv: "7d996b9617c286b8a1ce11e0cbd84002"
server_key_version: "encryptionkey@0"
I20260807 15:11:49.051234 6181 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115480851507-30472-0/minicluster-data/master-0/wal/instance:
uuid: "d526fffd10bd4d419a05f52b02285e6e"
format_stamp: "Formatted at 2026-08-07 15:11:49 on dist-test-slave-42z9"
server_key: "8b683a58ef3b2811b60d42932a919b76"
server_key_iv: "7d996b9617c286b8a1ce11e0cbd84002"
server_key_version: "encryptionkey@0"
I20260807 15:11:49.052681 6181 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 15:11:49.053803 6200 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:11:49.054039 6181 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:11:49.054136 6181 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115480851507-30472-0/minicluster-data/master-0/data,/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115480851507-30472-0/minicluster-data/master-0/wal
uuid: "d526fffd10bd4d419a05f52b02285e6e"
format_stamp: "Formatted at 2026-08-07 15:11:49 on dist-test-slave-42z9"
server_key: "8b683a58ef3b2811b60d42932a919b76"
server_key_iv: "7d996b9617c286b8a1ce11e0cbd84002"
server_key_version: "encryptionkey@0"
I20260807 15:11:49.054203 6181 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115480851507-30472-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115480851507-30472-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115480851507-30472-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:11:49.064312 6181 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:11:49.066948 6181 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:11:49.067086 6181 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:11:49.071805 6181 rpc_server.cc:307] RPC server started. Bound to: 127.29.194.62:38089
I20260807 15:11:49.071866 6277 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.194.62:38089 every 8 connection(s)
I20260807 15:11:49.072248 6181 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115480851507-30472-0/minicluster-data/master-0/data/info.pb
I20260807 15:11:49.073480 6282 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:11:49.076102 6282 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d526fffd10bd4d419a05f52b02285e6e: Bootstrap starting.
I20260807 15:11:49.076773 6282 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d526fffd10bd4d419a05f52b02285e6e: Neither blocks nor log segments found. Creating new log.
I20260807 15:11:49.077083 6282 log.cc:824] T 00000000000000000000000000000000 P d526fffd10bd4d419a05f52b02285e6e: Log is configured to *not* fsync() on all Append() calls
I20260807 15:11:49.078269 6282 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d526fffd10bd4d419a05f52b02285e6e: No bootstrap required, opened a new log
I20260807 15:11:49.080076 6282 raft_consensus.cc:348] T 00000000000000000000000000000000 P d526fffd10bd4d419a05f52b02285e6e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d526fffd10bd4d419a05f52b02285e6e" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 38089 } }
I20260807 15:11:49.080252 6282 raft_consensus.cc:374] T 00000000000000000000000000000000 P d526fffd10bd4d419a05f52b02285e6e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:11:49.080274 6282 raft_consensus.cc:729] T 00000000000000000000000000000000 P d526fffd10bd4d419a05f52b02285e6e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d526fffd10bd4d419a05f52b02285e6e, State: Initialized, Role: FOLLOWER
I20260807 15:11:49.080420 6282 consensus_queue.cc:260] T 00000000000000000000000000000000 P d526fffd10bd4d419a05f52b02285e6e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d526fffd10bd4d419a05f52b02285e6e" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 38089 } }
I20260807 15:11:49.080500 6282 raft_consensus.cc:388] T 00000000000000000000000000000000 P d526fffd10bd4d419a05f52b02285e6e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:11:49.080526 6282 raft_consensus.cc:482] T 00000000000000000000000000000000 P d526fffd10bd4d419a05f52b02285e6e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:11:49.080574 6282 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d526fffd10bd4d419a05f52b02285e6e [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:11:49.081276 6282 raft_consensus.cc:504] T 00000000000000000000000000000000 P d526fffd10bd4d419a05f52b02285e6e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d526fffd10bd4d419a05f52b02285e6e" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 38089 } }
I20260807 15:11:49.081383 6282 leader_election.cc:302] T 00000000000000000000000000000000 P d526fffd10bd4d419a05f52b02285e6e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d526fffd10bd4d419a05f52b02285e6e; no voters:
I20260807 15:11:49.081609 6282 leader_election.cc:288] T 00000000000000000000000000000000 P d526fffd10bd4d419a05f52b02285e6e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 15:11:49.081683 6287 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d526fffd10bd4d419a05f52b02285e6e [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:11:49.081913 6282 sys_catalog.cc:564] T 00000000000000000000000000000000 P d526fffd10bd4d419a05f52b02285e6e [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:11:49.082170 6287 raft_consensus.cc:686] T 00000000000000000000000000000000 P d526fffd10bd4d419a05f52b02285e6e [term 1 LEADER]: Becoming Leader. State: Replica: d526fffd10bd4d419a05f52b02285e6e, State: Running, Role: LEADER
I20260807 15:11:49.082417 30472 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7fZSIL/build/release/bin/kudu as pid 6181
I20260807 15:11:49.082645 30472 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115480851507-30472-0/minicluster-data/master-0/wal/instance
I20260807 15:11:49.083001 30472 external_mini_cluster.cc:1428] Setting key a1421072c511023b9c2768b900bbb15c
I20260807 15:11:49.083354 6287 consensus_queue.cc:237] T 00000000000000000000000000000000 P d526fffd10bd4d419a05f52b02285e6e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d526fffd10bd4d419a05f52b02285e6e" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 38089 } }
I20260807 15:11:49.084388 6288 sys_catalog.cc:455] T 00000000000000000000000000000000 P d526fffd10bd4d419a05f52b02285e6e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d526fffd10bd4d419a05f52b02285e6e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d526fffd10bd4d419a05f52b02285e6e" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 38089 } } }
I20260807 15:11:49.084517 6288 sys_catalog.cc:458] T 00000000000000000000000000000000 P d526fffd10bd4d419a05f52b02285e6e [sys.catalog]: This master's current role is: LEADER
I20260807 15:11:49.084626 6289 sys_catalog.cc:455] T 00000000000000000000000000000000 P d526fffd10bd4d419a05f52b02285e6e [sys.catalog]: SysCatalogTable state changed. Reason: New leader d526fffd10bd4d419a05f52b02285e6e. Latest consensus state: current_term: 1 leader_uuid: "d526fffd10bd4d419a05f52b02285e6e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d526fffd10bd4d419a05f52b02285e6e" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 38089 } } }
W20260807 15:11:49.085166 6307 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d526fffd10bd4d419a05f52b02285e6e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
I20260807 15:11:49.085215 6289 sys_catalog.cc:458] T 00000000000000000000000000000000 P d526fffd10bd4d419a05f52b02285e6e [sys.catalog]: This master's current role is: LEADER
W20260807 15:11:49.085216 6307 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 15:11:49.085357 6310 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:11:49.085579 6310 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:11:49.088490 6310 catalog_manager.cc:1347] Generated new cluster ID: 38519932028840ac871d977fe516f450
I20260807 15:11:49.088577 6310 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 07 15:11:49 dist-test-slave-42z9 krb5kdc[6163](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115508, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.194.62@KRBTEST.COM
I20260807 15:11:49.095386 6305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:49.083577 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52244 (local address 127.29.194.62:38089)
0807 15:11:49.083750 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:49.083760 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:49.083863 (+ 103us) server_negotiation.cc:407] Connection header received
0807 15:11:49.084046 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:49.084054 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:49.084149 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:49.084322 (+ 173us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:11:49.086243 (+ 1921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:49.087616 (+ 1373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:49.088851 (+ 1235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:49.089341 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:49.089476 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:49.092049 (+ 2573us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:11:49.092055 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:11:49.092056 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:11:49.092071 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:11:49.093838 (+ 1767us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:49.094203 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:49.094206 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:49.094206 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:49.094300 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:49.094482 (+ 182us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:49.094483 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:49.094484 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:49.094753 (+ 269us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:11:49.094918 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0807 15:11:49.095045 (+ 127us) server_negotiation.cc:299] Negotiation successful
0807 15:11:49.095198 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":114,"thread_start_us":83,"threads_started":1}
I20260807 15:11:49.103109 6310 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:11:49.104086 6310 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:11:49.108068 6310 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d526fffd10bd4d419a05f52b02285e6e: Generated new TSK 0
I20260807 15:11:49.108299 6310 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.29.194.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.194.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.194.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.1.keytab.
Entry for principal kudu/127.29.194.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.1.keytab.
I20260807 15:11:49.147140 30472 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7fZSIL/build/release/bin/kudu
/tmp/dist-test-task7fZSIL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115480851507-30472-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115480851507-30472-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115480851507-30472-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115480851507-30472-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.194.1:0
--local_ip_for_outbound_sockets=127.29.194.1
--webserver_interface=127.29.194.1
--webserver_port=0
--tserver_master_addrs=127.29.194.62:38089
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.1.keytab
--principal=kudu/127.29.194.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.194.20:43329
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115480851507-30472-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115480851507-30472-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115480851507-30472-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:11:49.244823 6320 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:11:49.245056 6320 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:11:49.245085 6320 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:11:49.246454 6320 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:11:49.246508 6320 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:11:49.246539 6320 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.194.1
I20260807 15:11:49.248253 6320 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.194.20:43329
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115480851507-30472-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115480851507-30472-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.1.keytab
--principal=kudu/127.29.194.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.194.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115480851507-30472-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.194.1
--webserver_port=0
--tserver_master_addrs=127.29.194.62:38089
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6320
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.194.1
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115480851507-30472-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:11:49.248553 6320 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:11:49.248806 6320 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:11:49.254570 6328 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:11:49.254570 6330 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:11:49.254819 6320 server_base.cc:1034] running on GCE node
W20260807 15:11:49.254570 6332 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:11:49.255100 6320 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:11:49.255352 6320 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:11:49.256523 6320 hybrid_clock.cc:648] HybridClock initialized: now 1786115509256512 us; error 30 us; skew 500 ppm
Aug 07 15:11:49 dist-test-slave-42z9 krb5kdc[6163](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115509, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.194.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:11:49.260062 6320 init.cc:377] Logged in from keytab as kudu/127.29.194.1@KRBTEST.COM (short username kudu)
I20260807 15:11:49.260713 6320 webserver.cc:460] Webserver started at http://127.29.194.1:35883/ using document root <none> and password file <none>
I20260807 15:11:49.260988 6320 fs_manager.cc:359] Metadata directory not provided
I20260807 15:11:49.261036 6320 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:11:49.261157 6320 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:11:49.262739 6320 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115480851507-30472-0/minicluster-data/ts-0/data/instance:
uuid: "81bd34cf9792455f9416796f226c548f"
format_stamp: "Formatted at 2026-08-07 15:11:49 on dist-test-slave-42z9"
server_key: "f4ddac7fdd49779df1af6695a158e870"
server_key_iv: "4a08f0c14888fe639c101d5faf3698f0"
server_key_version: "encryptionkey@0"
I20260807 15:11:49.263137 6320 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115480851507-30472-0/minicluster-data/ts-0/wal/instance:
uuid: "81bd34cf9792455f9416796f226c548f"
format_stamp: "Formatted at 2026-08-07 15:11:49 on dist-test-slave-42z9"
server_key: "f4ddac7fdd49779df1af6695a158e870"
server_key_iv: "4a08f0c14888fe639c101d5faf3698f0"
server_key_version: "encryptionkey@0"
I20260807 15:11:49.264668 6320 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260807 15:11:49.265748 6343 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:11:49.265940 6320 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 15:11:49.266038 6320 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115480851507-30472-0/minicluster-data/ts-0/data,/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115480851507-30472-0/minicluster-data/ts-0/wal
uuid: "81bd34cf9792455f9416796f226c548f"
format_stamp: "Formatted at 2026-08-07 15:11:49 on dist-test-slave-42z9"
server_key: "f4ddac7fdd49779df1af6695a158e870"
server_key_iv: "4a08f0c14888fe639c101d5faf3698f0"
server_key_version: "encryptionkey@0"
I20260807 15:11:49.266105 6320 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115480851507-30472-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115480851507-30472-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115480851507-30472-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:11:49.279881 6320 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:11:49.282374 6320 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:11:49.282553 6320 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:11:49.282820 6320 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:11:49.283223 6320 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:11:49.283260 6320 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:11:49.283293 6320 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:11:49.283309 6320 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:11:49.289189 6320 rpc_server.cc:307] RPC server started. Bound to: 127.29.194.1:39547
I20260807 15:11:49.289418 6504 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.194.1:39547 every 8 connection(s)
I20260807 15:11:49.289708 6320 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115480851507-30472-0/minicluster-data/ts-0/data/info.pb
I20260807 15:11:49.295763 30472 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7fZSIL/build/release/bin/kudu as pid 6320
I20260807 15:11:49.295943 30472 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115480851507-30472-0/minicluster-data/ts-0/wal/instance
I20260807 15:11:49.296182 30472 external_mini_cluster.cc:1428] Setting key def78655f7635db7db854cbf8b72c25a
Aug 07 15:11:49 dist-test-slave-42z9 krb5kdc[6163](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115509, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.194.1@KRBTEST.COM for kudu/127.29.194.62@KRBTEST.COM
I20260807 15:11:49.300251 6305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:49.290738 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.194.1:53739 (local address 127.29.194.62:38089)
0807 15:11:49.290840 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:49.290842 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:49.291066 (+ 224us) server_negotiation.cc:407] Connection header received
0807 15:11:49.292106 (+ 1040us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:49.292107 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:49.292126 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:49.292181 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:11:49.292903 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:49.293423 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:49.294479 (+ 1056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:49.294689 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:49.294748 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:49.297498 (+ 2750us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:11:49.297500 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:11:49.297500 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:11:49.297508 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:11:49.298818 (+ 1310us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:49.299260 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:49.299261 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:49.299261 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:49.299325 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:49.299583 (+ 258us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:49.299584 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:49.299584 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:49.299708 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:11:49.299765 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0807 15:11:49.300050 (+ 285us) server_negotiation.cc:299] Negotiation successful
0807 15:11:49.300159 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260807 15:11:49.300436 6509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:49.290678 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:38089 (local address 127.29.194.1:53739)
0807 15:11:49.290869 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0807 15:11:49.290958 (+ 89us) client_negotiation.cc:174] Beginning negotiation
0807 15:11:49.291971 (+ 1013us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:11:49.292285 (+ 314us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:11:49.292288 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:11:49.292587 (+ 299us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:11:49.292796 (+ 209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:49.292800 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:49.293547 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:49.293548 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:49.294388 (+ 840us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:49.294390 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:49.294766 (+ 376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:49.294767 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:49.294901 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:49.295582 (+ 681us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:11:49.295601 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:11:49.297360 (+ 1759us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:11:49.298909 (+ 1549us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:11:49.298919 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:11:49.298922 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:11:49.299186 (+ 264us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:11:49.299403 (+ 217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:11:49.299404 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:11:49.299404 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:11:49.299521 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:11:49.299768 (+ 247us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:11:49.299776 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:11:49.299924 (+ 148us) client_negotiation.cc:769] Sending connection context
0807 15:11:49.300020 (+ 96us) client_negotiation.cc:240] Negotiation successful
0807 15:11:49.300203 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":87,"threads_started":1}
I20260807 15:11:49.300895 6507 heartbeater.cc:347] Connected to a master server at 127.29.194.62:38089
I20260807 15:11:49.300997 6507 heartbeater.cc:464] Registering TS with master...
I20260807 15:11:49.301192 6507 heartbeater.cc:511] Master 127.29.194.62:38089 requested a full tablet report, sending...
I20260807 15:11:49.301740 6223 ts_manager.cc:194] Registered new tserver with Master: 81bd34cf9792455f9416796f226c548f (127.29.194.1:39547)
I20260807 15:11:49.302793 6223 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.194.1@KRBTEST.COM'} at 127.29.194.1:53739
WARNING: no policy specified for kudu/127.29.194.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.194.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.194.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.2.keytab.
Entry for principal kudu/127.29.194.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.2.keytab.
I20260807 15:11:49.329653 30472 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7fZSIL/build/release/bin/kudu
/tmp/dist-test-task7fZSIL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115480851507-30472-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115480851507-30472-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115480851507-30472-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115480851507-30472-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.194.2:0
--local_ip_for_outbound_sockets=127.29.194.2
--webserver_interface=127.29.194.2
--webserver_port=0
--tserver_master_addrs=127.29.194.62:38089
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.2.keytab
--principal=kudu/127.29.194.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.194.20:43329
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115480851507-30472-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115480851507-30472-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115480851507-30472-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:11:49.420145 6513 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:11:49.420395 6513 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:11:49.420423 6513 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:11:49.421736 6513 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:11:49.421797 6513 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:11:49.421826 6513 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.194.2
I20260807 15:11:49.423501 6513 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.194.20:43329
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115480851507-30472-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115480851507-30472-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.2.keytab
--principal=kudu/127.29.194.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.194.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115480851507-30472-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.194.2
--webserver_port=0
--tserver_master_addrs=127.29.194.62:38089
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6513
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.194.2
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115480851507-30472-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:11:49.424000 6513 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:11:49.424376 6513 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:11:49.431267 6520 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:11:49.431226 6521 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:11:49.431269 6523 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:11:49.432382 6513 server_base.cc:1034] running on GCE node
I20260807 15:11:49.432631 6513 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:11:49.432878 6513 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:11:49.434195 6513 hybrid_clock.cc:648] HybridClock initialized: now 1786115509434062 us; error 46 us; skew 500 ppm
Aug 07 15:11:49 dist-test-slave-42z9 krb5kdc[6163](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115509, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.194.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:11:49.437597 6513 init.cc:377] Logged in from keytab as kudu/127.29.194.2@KRBTEST.COM (short username kudu)
I20260807 15:11:49.438330 6513 webserver.cc:460] Webserver started at http://127.29.194.2:39889/ using document root <none> and password file <none>
I20260807 15:11:49.438629 6513 fs_manager.cc:359] Metadata directory not provided
I20260807 15:11:49.438691 6513 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:11:49.438861 6513 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:11:49.440202 6513 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115480851507-30472-0/minicluster-data/ts-1/data/instance:
uuid: "64121e6f4cae4d4b9b59c373aa149b1d"
format_stamp: "Formatted at 2026-08-07 15:11:49 on dist-test-slave-42z9"
server_key: "6177545b5797d3bf9b6f6eb7d34fdf6d"
server_key_iv: "2d9594f946d50ae96d33cbee12126cf5"
server_key_version: "encryptionkey@0"
I20260807 15:11:49.440634 6513 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115480851507-30472-0/minicluster-data/ts-1/wal/instance:
uuid: "64121e6f4cae4d4b9b59c373aa149b1d"
format_stamp: "Formatted at 2026-08-07 15:11:49 on dist-test-slave-42z9"
server_key: "6177545b5797d3bf9b6f6eb7d34fdf6d"
server_key_iv: "2d9594f946d50ae96d33cbee12126cf5"
server_key_version: "encryptionkey@0"
I20260807 15:11:49.442454 6513 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.004s sys 0.000s
I20260807 15:11:49.443516 6532 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:11:49.444020 6513 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:11:49.444160 6513 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115480851507-30472-0/minicluster-data/ts-1/data,/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115480851507-30472-0/minicluster-data/ts-1/wal
uuid: "64121e6f4cae4d4b9b59c373aa149b1d"
format_stamp: "Formatted at 2026-08-07 15:11:49 on dist-test-slave-42z9"
server_key: "6177545b5797d3bf9b6f6eb7d34fdf6d"
server_key_iv: "2d9594f946d50ae96d33cbee12126cf5"
server_key_version: "encryptionkey@0"
I20260807 15:11:49.444252 6513 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115480851507-30472-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115480851507-30472-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115480851507-30472-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:11:49.452889 6513 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:11:49.455492 6513 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:11:49.455659 6513 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:11:49.455904 6513 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:11:49.456236 6513 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:11:49.456272 6513 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:11:49.456303 6513 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:11:49.456322 6513 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:11:49.463250 6513 rpc_server.cc:307] RPC server started. Bound to: 127.29.194.2:39385
I20260807 15:11:49.463375 6721 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.194.2:39385 every 8 connection(s)
I20260807 15:11:49.463693 6513 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115480851507-30472-0/minicluster-data/ts-1/data/info.pb
I20260807 15:11:49.466483 30472 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7fZSIL/build/release/bin/kudu as pid 6513
I20260807 15:11:49.466616 30472 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115480851507-30472-0/minicluster-data/ts-1/wal/instance
I20260807 15:11:49.467022 30472 external_mini_cluster.cc:1428] Setting key 4b5d7e717dbdf995b145449df965f547
Aug 07 15:11:49 dist-test-slave-42z9 krb5kdc[6163](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115509, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.194.2@KRBTEST.COM for kudu/127.29.194.62@KRBTEST.COM
I20260807 15:11:49.476856 6305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:49.464671 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.194.2:47887 (local address 127.29.194.62:38089)
0807 15:11:49.464732 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:49.464735 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:49.464987 (+ 252us) server_negotiation.cc:407] Connection header received
0807 15:11:49.466586 (+ 1599us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:49.466589 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:49.466610 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:49.466692 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:11:49.467911 (+ 1219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:49.468586 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:49.470192 (+ 1606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:49.470484 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:49.470571 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:49.473471 (+ 2900us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:11:49.473474 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:11:49.473474 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:11:49.473483 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:11:49.475063 (+ 1580us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:49.475652 (+ 589us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:49.475653 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:49.475653 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:49.475724 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:49.476025 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:49.476026 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:49.476026 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:49.476180 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:11:49.476249 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0807 15:11:49.476639 (+ 390us) server_negotiation.cc:299] Negotiation successful
0807 15:11:49.476752 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
WARNING: no policy specified for kudu/127.29.194.3@KRBTEST.COM; defaulting to no policy
I20260807 15:11:49.477090 6725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:49.464584 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:38089 (local address 127.29.194.2:47887)
0807 15:11:49.464811 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0807 15:11:49.464858 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0807 15:11:49.466364 (+ 1506us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:11:49.466875 (+ 511us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:11:49.466881 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:11:49.467347 (+ 466us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:11:49.467703 (+ 356us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:49.467712 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:49.468717 (+ 1005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:49.468719 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:49.470076 (+ 1357us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:49.470090 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:49.470584 (+ 494us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:49.470586 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:49.470745 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:49.471346 (+ 601us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:11:49.471357 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:11:49.473309 (+ 1952us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:11:49.475175 (+ 1866us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:11:49.475188 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:11:49.475192 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:11:49.475553 (+ 361us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:11:49.475795 (+ 242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:11:49.475796 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:11:49.475797 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:11:49.475952 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:11:49.476256 (+ 304us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:11:49.476264 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:11:49.476468 (+ 204us) client_negotiation.cc:769] Sending connection context
0807 15:11:49.476609 (+ 141us) client_negotiation.cc:240] Negotiation successful
0807 15:11:49.476811 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":90,"threads_started":1}
I20260807 15:11:49.478091 6722 heartbeater.cc:347] Connected to a master server at 127.29.194.62:38089
I20260807 15:11:49.478204 6722 heartbeater.cc:464] Registering TS with master...
I20260807 15:11:49.478374 6722 heartbeater.cc:511] Master 127.29.194.62:38089 requested a full tablet report, sending...
I20260807 15:11:49.479497 6223 ts_manager.cc:194] Registered new tserver with Master: 64121e6f4cae4d4b9b59c373aa149b1d (127.29.194.2:39385)
I20260807 15:11:49.479919 6223 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.194.2@KRBTEST.COM'} at 127.29.194.2:47887
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.194.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.194.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.3.keytab.
Entry for principal kudu/127.29.194.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.3.keytab.
I20260807 15:11:49.506906 30472 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7fZSIL/build/release/bin/kudu
/tmp/dist-test-task7fZSIL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115480851507-30472-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115480851507-30472-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115480851507-30472-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115480851507-30472-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.194.3:0
--local_ip_for_outbound_sockets=127.29.194.3
--webserver_interface=127.29.194.3
--webserver_port=0
--tserver_master_addrs=127.29.194.62:38089
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.3.keytab
--principal=kudu/127.29.194.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.194.20:43329
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115480851507-30472-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115480851507-30472-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115480851507-30472-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:11:49.599822 6728 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:11:49.600071 6728 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:11:49.600097 6728 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:11:49.601316 6728 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:11:49.601367 6728 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:11:49.601394 6728 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.194.3
I20260807 15:11:49.603269 6728 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.194.20:43329
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115480851507-30472-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115480851507-30472-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.3.keytab
--principal=kudu/127.29.194.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.194.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115480851507-30472-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.194.3
--webserver_port=0
--tserver_master_addrs=127.29.194.62:38089
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6728
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.194.3
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115480851507-30472-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:11:49.603530 6728 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:11:49.603741 6728 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:11:49.609174 6736 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:11:49.609163 6737 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:11:49.609164 6739 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:11:49.610379 6728 server_base.cc:1034] running on GCE node
I20260807 15:11:49.610652 6728 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:11:49.610909 6728 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:11:49.612092 6728 hybrid_clock.cc:648] HybridClock initialized: now 1786115509612064 us; error 72 us; skew 500 ppm
Aug 07 15:11:49 dist-test-slave-42z9 krb5kdc[6163](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115509, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.194.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:11:49.614967 6728 init.cc:377] Logged in from keytab as kudu/127.29.194.3@KRBTEST.COM (short username kudu)
I20260807 15:11:49.615556 6728 webserver.cc:460] Webserver started at http://127.29.194.3:46111/ using document root <none> and password file <none>
I20260807 15:11:49.615796 6728 fs_manager.cc:359] Metadata directory not provided
I20260807 15:11:49.615849 6728 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:11:49.615974 6728 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:11:49.617038 6728 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115480851507-30472-0/minicluster-data/ts-2/data/instance:
uuid: "c39537ca8a2e46a2a8205cd25a5eacc2"
format_stamp: "Formatted at 2026-08-07 15:11:49 on dist-test-slave-42z9"
server_key: "382eb65f2141936269dc6f47d3a3a574"
server_key_iv: "fd8dc00505a13ea8dfcedb015555374c"
server_key_version: "encryptionkey@0"
I20260807 15:11:49.617388 6728 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115480851507-30472-0/minicluster-data/ts-2/wal/instance:
uuid: "c39537ca8a2e46a2a8205cd25a5eacc2"
format_stamp: "Formatted at 2026-08-07 15:11:49 on dist-test-slave-42z9"
server_key: "382eb65f2141936269dc6f47d3a3a574"
server_key_iv: "fd8dc00505a13ea8dfcedb015555374c"
server_key_version: "encryptionkey@0"
I20260807 15:11:49.618963 6728 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:11:49.620052 6749 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:11:49.620257 6728 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:11:49.620425 6728 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115480851507-30472-0/minicluster-data/ts-2/data,/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115480851507-30472-0/minicluster-data/ts-2/wal
uuid: "c39537ca8a2e46a2a8205cd25a5eacc2"
format_stamp: "Formatted at 2026-08-07 15:11:49 on dist-test-slave-42z9"
server_key: "382eb65f2141936269dc6f47d3a3a574"
server_key_iv: "fd8dc00505a13ea8dfcedb015555374c"
server_key_version: "encryptionkey@0"
I20260807 15:11:49.620520 6728 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115480851507-30472-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115480851507-30472-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115480851507-30472-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:11:49.638129 6728 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:11:49.641861 6728 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:11:49.642043 6728 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:11:49.642314 6728 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:11:49.642661 6728 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:11:49.642696 6728 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:11:49.642727 6728 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:11:49.642752 6728 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:11:49.649351 6728 rpc_server.cc:307] RPC server started. Bound to: 127.29.194.3:35305
I20260807 15:11:49.649430 6917 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.194.3:35305 every 8 connection(s)
I20260807 15:11:49.649783 6728 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115480851507-30472-0/minicluster-data/ts-2/data/info.pb
I20260807 15:11:49.653894 30472 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7fZSIL/build/release/bin/kudu as pid 6728
I20260807 15:11:49.654024 30472 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115480851507-30472-0/minicluster-data/ts-2/wal/instance
I20260807 15:11:49.654249 30472 external_mini_cluster.cc:1428] Setting key 12049c750b6bb94843f6456df9898f5e
Aug 07 15:11:49 dist-test-slave-42z9 krb5kdc[6163](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115509, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.194.3@KRBTEST.COM for kudu/127.29.194.62@KRBTEST.COM
I20260807 15:11:49.661651 6305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:49.650577 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.194.3:60983 (local address 127.29.194.62:38089)
0807 15:11:49.650658 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:49.650660 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:49.650826 (+ 166us) server_negotiation.cc:407] Connection header received
0807 15:11:49.651842 (+ 1016us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:49.651844 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:49.651859 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:49.651913 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:11:49.652616 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:49.653118 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:49.654216 (+ 1098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:49.654429 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:49.654486 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:49.657846 (+ 3360us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:11:49.657848 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:11:49.657849 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:11:49.657861 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:11:49.659504 (+ 1643us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:49.660131 (+ 627us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:49.660132 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:49.660133 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:49.660248 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:49.660664 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:49.660666 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:49.660666 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:49.660833 (+ 167us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:11:49.660909 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0807 15:11:49.661371 (+ 462us) server_negotiation.cc:299] Negotiation successful
0807 15:11:49.661570 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260807 15:11:49.661898 6923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:49.650525 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:38089 (local address 127.29.194.3:60983)
0807 15:11:49.650704 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0807 15:11:49.650744 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0807 15:11:49.651722 (+ 978us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:11:49.652005 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:11:49.652009 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:11:49.652305 (+ 296us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:11:49.652510 (+ 205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:49.652516 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:49.653208 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:49.653209 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:49.654133 (+ 924us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:49.654136 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:49.654518 (+ 382us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:49.654519 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:49.654678 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:49.655400 (+ 722us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:11:49.655417 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:11:49.657587 (+ 2170us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:11:49.659647 (+ 2060us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:11:49.659661 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:11:49.659664 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:11:49.660007 (+ 343us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:11:49.660376 (+ 369us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:11:49.660378 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:11:49.660378 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:11:49.660550 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:11:49.660977 (+ 427us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:11:49.660985 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:11:49.661166 (+ 181us) client_negotiation.cc:769] Sending connection context
0807 15:11:49.661305 (+ 139us) client_negotiation.cc:240] Negotiation successful
0807 15:11:49.661558 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":81,"threads_started":1}
I20260807 15:11:49.662714 6920 heartbeater.cc:347] Connected to a master server at 127.29.194.62:38089
I20260807 15:11:49.662891 6920 heartbeater.cc:464] Registering TS with master...
I20260807 15:11:49.663136 6920 heartbeater.cc:511] Master 127.29.194.62:38089 requested a full tablet report, sending...
I20260807 15:11:49.663690 6223 ts_manager.cc:194] Registered new tserver with Master: c39537ca8a2e46a2a8205cd25a5eacc2 (127.29.194.3:35305)
I20260807 15:11:49.664140 6223 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.194.3@KRBTEST.COM'} at 127.29.194.3:60983
I20260807 15:11:49.667227 30472 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115480851507-30472-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115480851507-30472-0/krb5kdc/test-admin.keytab.
Aug 07 15:11:49 dist-test-slave-42z9 krb5kdc[6163](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115509, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:11:49.687971 30472 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260807 15:11:49.690231 6928 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:11:49.689024 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55866 (local address 127.29.194.1:39547)
0807 15:11:49.689248 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:49.689253 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:49.689264 (+ 11us) server_negotiation.cc:407] Connection header received
0807 15:11:49.689293 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:49.689295 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:49.689309 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:49.689382 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:11:49.690158 (+ 776us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55866: BlockingRecv error: recv got EOF from 127.0.0.1:55866 (error 108)
Metrics: {"server-negotiator.queue_time_us":190,"thread_start_us":141,"threads_started":1}
W20260807 15:11:49.690583 30472 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115480851507-30472-0/krb5kdc/krb5cc' of type FILE
Aug 07 15:11:49 dist-test-slave-42z9 krb5kdc[6163](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115509, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:11:49.691834 30472 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 07 15:11:49 dist-test-slave-42z9 krb5kdc[6163](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115509, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:11:49.692956 30472 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 07 15:11:49 dist-test-slave-42z9 krb5kdc[6163](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115509, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.194.1@KRBTEST.COM
I20260807 15:11:49.703634 6928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:49.693712 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55870 (local address 127.29.194.1:39547)
0807 15:11:49.693776 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:49.693778 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:49.693809 (+ 31us) server_negotiation.cc:407] Connection header received
0807 15:11:49.693905 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:49.693906 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:49.693919 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:49.693970 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:11:49.694655 (+ 685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:49.695451 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:49.696282 (+ 831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:49.696636 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:49.696701 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:49.699713 (+ 3012us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:11:49.699727 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:11:49.699727 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:11:49.699737 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:11:49.701761 (+ 2024us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:49.702335 (+ 574us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:49.702341 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:49.702341 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:49.702435 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:49.702770 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:49.702771 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:49.702772 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:49.703012 (+ 240us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:11:49.703186 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0807 15:11:49.703375 (+ 189us) server_negotiation.cc:299] Negotiation successful
0807 15:11:49.703504 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
W20260807 15:11:49.706560 6928 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:11:49.705667 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55872 (local address 127.29.194.1:39547)
0807 15:11:49.705819 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:49.705821 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:49.705831 (+ 10us) server_negotiation.cc:407] Connection header received
0807 15:11:49.705855 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:49.705856 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:49.705880 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:49.706007 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:11:49.706501 (+ 494us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55872: BlockingRecv error: recv got EOF from 127.0.0.1:55872 (error 108)
Metrics: {"server-negotiator.queue_time_us":121}
W20260807 15:11:49.706722 30472 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115480851507-30472-0/krb5kdc/krb5cc' of type FILE
Aug 07 15:11:49 dist-test-slave-42z9 krb5kdc[6163](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115509, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:11:49.707947 30472 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 07 15:11:49 dist-test-slave-42z9 krb5kdc[6163](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115509, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:11:49.708866 30472 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 07 15:11:49 dist-test-slave-42z9 krb5kdc[6163](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115509, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.194.1@KRBTEST.COM
I20260807 15:11:49.717301 6928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:49.709541 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55882 (local address 127.29.194.1:39547)
0807 15:11:49.709602 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:49.709604 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:49.709664 (+ 60us) server_negotiation.cc:407] Connection header received
0807 15:11:49.709734 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:49.709735 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:49.709748 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:49.709789 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:11:49.710251 (+ 462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:49.710801 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:49.711577 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:49.711783 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:49.711846 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:49.714412 (+ 2566us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:11:49.714414 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:11:49.714414 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:11:49.714422 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:11:49.715744 (+ 1322us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:49.716184 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:49.716185 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:49.716186 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:49.716244 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:49.716525 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:49.716528 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:49.716528 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:49.716727 (+ 199us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:11:49.716854 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0807 15:11:49.717041 (+ 187us) server_negotiation.cc:299] Negotiation successful
0807 15:11:49.717180 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
W20260807 15:11:49.719686 6928 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:11:49.718904 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55884 (local address 127.29.194.1:39547)
0807 15:11:49.718966 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:49.718968 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:49.719059 (+ 91us) server_negotiation.cc:407] Connection header received
0807 15:11:49.719107 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:49.719108 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:49.719117 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:49.719173 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:11:49.719623 (+ 450us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55884: BlockingRecv error: recv got EOF from 127.0.0.1:55884 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
W20260807 15:11:49.719879 30472 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115480851507-30472-0/krb5kdc/krb5cc' of type FILE
Aug 07 15:11:49 dist-test-slave-42z9 krb5kdc[6163](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115509, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:11:49.721197 30472 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 07 15:11:49 dist-test-slave-42z9 krb5kdc[6163](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115509, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:11:49.722229 30472 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 07 15:11:49 dist-test-slave-42z9 krb5kdc[6163](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115509, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.194.1@KRBTEST.COM
I20260807 15:11:49.730671 6928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:49.723010 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55898 (local address 127.29.194.1:39547)
0807 15:11:49.723168 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:49.723170 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:49.723209 (+ 39us) server_negotiation.cc:407] Connection header received
0807 15:11:49.723235 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:49.723236 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:49.723247 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:49.723290 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:11:49.723784 (+ 494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:49.724397 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:49.725217 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:49.725705 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:49.725811 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:49.728214 (+ 2403us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:11:49.728215 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:11:49.728216 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:11:49.728224 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:11:49.729667 (+ 1443us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:49.730000 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:49.730001 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:49.730002 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:49.730077 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:49.730222 (+ 145us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:49.730222 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:49.730223 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:49.730327 (+ 104us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:11:49.730399 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0807 15:11:49.730507 (+ 108us) server_negotiation.cc:299] Negotiation successful
0807 15:11:49.730582 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94}
I20260807 15:11:49.731354 30472 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7fZSIL/build/release/bin/kudu with pid 6320
I20260807 15:11:49.743989 30472 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7fZSIL/build/release/bin/kudu with pid 6513
I20260807 15:11:49.761857 30472 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7fZSIL/build/release/bin/kudu with pid 6728
I20260807 15:11:49.776346 30472 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7fZSIL/build/release/bin/kudu with pid 6181
2026-08-07T15:11:49Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (2165 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115480851507-30472-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 15:11:49 dist-test-slave-42z9 krb5kdc[6944](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 15:11:49 dist-test-slave-42z9 krb5kdc[6944](info): set up 2 sockets
Aug 07 15:11:49 dist-test-slave-42z9 krb5kdc[6944](info): commencing operation
krb5kdc: starting...
W20260807 15:11:50.954605 30472 mini_kdc.cc:121] Time spent starting KDC: real 1.150s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115480851507-30472-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115480851507-30472-0/krb5kdc/test-user.keytab.
Aug 07 15:11:51 dist-test-slave-42z9 krb5kdc[6944](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115511, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T15:11:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T15:11:51Z Disabled control of system clock
WARNING: no policy specified for oryx/127.29.194.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.29.194.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.29.194.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115480851507-30472-0/krb5kdc/oryx_127.29.194.62.keytab.
Entry for principal oryx/127.29.194.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115480851507-30472-0/krb5kdc/oryx_127.29.194.62.keytab.
I20260807 15:11:51.107128 30472 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7fZSIL/build/release/bin/kudu
/tmp/dist-test-task7fZSIL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115480851507-30472-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115480851507-30472-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115480851507-30472-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115480851507-30472-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.194.62:39853
--webserver_interface=127.29.194.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115480851507-30472-0/krb5kdc/oryx_127.29.194.62.keytab
--principal=oryx/127.29.194.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.194.20:45569
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.194.62:39853
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115480851507-30472-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115480851507-30472-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115480851507-30472-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:11:51.200953 6965 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:11:51.201215 6965 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:11:51.201241 6965 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:11:51.202595 6965 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:11:51.202659 6965 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:11:51.202673 6965 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:11:51.202684 6965 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:11:51.202696 6965 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:11:51.204433 6965 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.194.20:45569
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115480851507-30472-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115480851507-30472-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.194.62:39853
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115480851507-30472-0/krb5kdc/oryx_127.29.194.62.keytab
--principal=oryx/127.29.194.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.194.62:39853
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115480851507-30472-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.194.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6965
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115480851507-30472-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:11:51.204720 6965 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:11:51.204993 6965 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:11:51.210750 6972 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:11:51.210925 6965 server_base.cc:1034] running on GCE node
W20260807 15:11:51.210731 6971 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:11:51.210731 6974 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:11:51.211488 6965 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:11:51.211809 6965 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:11:51.213055 6965 hybrid_clock.cc:648] HybridClock initialized: now 1786115511213019 us; error 41 us; skew 500 ppm
Aug 07 15:11:51 dist-test-slave-42z9 krb5kdc[6944](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115511, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.194.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:11:51.216610 6965 init.cc:377] Logged in from keytab as oryx/127.29.194.62@KRBTEST.COM (short username oryx)
I20260807 15:11:51.217286 6965 webserver.cc:460] Webserver started at http://127.29.194.62:37891/ using document root <none> and password file <none>
I20260807 15:11:51.217567 6965 fs_manager.cc:359] Metadata directory not provided
I20260807 15:11:51.217622 6965 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:11:51.217761 6965 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:11:51.219180 6965 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115480851507-30472-0/minicluster-data/master-0/data/instance:
uuid: "4ebd9481251f4a29ba10f88819ba9ae2"
format_stamp: "Formatted at 2026-08-07 15:11:51 on dist-test-slave-42z9"
server_key: "dfaf67a7c94483f01825b36aea3bbd58"
server_key_iv: "e343faf8868a4e388b15267c83dd65ac"
server_key_version: "encryptionkey@0"
I20260807 15:11:51.219622 6965 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115480851507-30472-0/minicluster-data/master-0/wal/instance:
uuid: "4ebd9481251f4a29ba10f88819ba9ae2"
format_stamp: "Formatted at 2026-08-07 15:11:51 on dist-test-slave-42z9"
server_key: "dfaf67a7c94483f01825b36aea3bbd58"
server_key_iv: "e343faf8868a4e388b15267c83dd65ac"
server_key_version: "encryptionkey@0"
I20260807 15:11:51.221079 6965 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 15:11:51.222047 6983 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:11:51.222240 6965 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260807 15:11:51.222335 6965 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115480851507-30472-0/minicluster-data/master-0/data,/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115480851507-30472-0/minicluster-data/master-0/wal
uuid: "4ebd9481251f4a29ba10f88819ba9ae2"
format_stamp: "Formatted at 2026-08-07 15:11:51 on dist-test-slave-42z9"
server_key: "dfaf67a7c94483f01825b36aea3bbd58"
server_key_iv: "e343faf8868a4e388b15267c83dd65ac"
server_key_version: "encryptionkey@0"
I20260807 15:11:51.222409 6965 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115480851507-30472-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115480851507-30472-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115480851507-30472-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:11:51.240177 6965 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:11:51.242743 6965 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:11:51.242874 6965 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:11:51.247124 6965 rpc_server.cc:307] RPC server started. Bound to: 127.29.194.62:39853
I20260807 15:11:51.247291 7053 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.194.62:39853 every 8 connection(s)
I20260807 15:11:51.247644 6965 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115480851507-30472-0/minicluster-data/master-0/data/info.pb
I20260807 15:11:51.249235 7054 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:11:51.251958 7054 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4ebd9481251f4a29ba10f88819ba9ae2: Bootstrap starting.
I20260807 15:11:51.252601 7054 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4ebd9481251f4a29ba10f88819ba9ae2: Neither blocks nor log segments found. Creating new log.
I20260807 15:11:51.252895 7054 log.cc:824] T 00000000000000000000000000000000 P 4ebd9481251f4a29ba10f88819ba9ae2: Log is configured to *not* fsync() on all Append() calls
I20260807 15:11:51.254092 7054 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4ebd9481251f4a29ba10f88819ba9ae2: No bootstrap required, opened a new log
I20260807 15:11:51.254112 30472 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7fZSIL/build/release/bin/kudu as pid 6965
I20260807 15:11:51.254211 30472 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115480851507-30472-0/minicluster-data/master-0/wal/instance
I20260807 15:11:51.254422 30472 external_mini_cluster.cc:1428] Setting key f5854d8de36ea9da320f9940c0119772
I20260807 15:11:51.256143 7054 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4ebd9481251f4a29ba10f88819ba9ae2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ebd9481251f4a29ba10f88819ba9ae2" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 39853 } }
I20260807 15:11:51.256299 7054 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4ebd9481251f4a29ba10f88819ba9ae2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:11:51.256326 7054 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4ebd9481251f4a29ba10f88819ba9ae2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ebd9481251f4a29ba10f88819ba9ae2, State: Initialized, Role: FOLLOWER
I20260807 15:11:51.256469 7054 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4ebd9481251f4a29ba10f88819ba9ae2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ebd9481251f4a29ba10f88819ba9ae2" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 39853 } }
I20260807 15:11:51.256579 7054 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4ebd9481251f4a29ba10f88819ba9ae2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:11:51.256626 7054 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4ebd9481251f4a29ba10f88819ba9ae2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:11:51.256681 7054 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4ebd9481251f4a29ba10f88819ba9ae2 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:11:51.257859 7054 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4ebd9481251f4a29ba10f88819ba9ae2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ebd9481251f4a29ba10f88819ba9ae2" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 39853 } }
I20260807 15:11:51.258023 7054 leader_election.cc:302] T 00000000000000000000000000000000 P 4ebd9481251f4a29ba10f88819ba9ae2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4ebd9481251f4a29ba10f88819ba9ae2; no voters:
I20260807 15:11:51.258244 7054 leader_election.cc:288] T 00000000000000000000000000000000 P 4ebd9481251f4a29ba10f88819ba9ae2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 15:11:51.258302 7062 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4ebd9481251f4a29ba10f88819ba9ae2 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:11:51.258554 7062 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4ebd9481251f4a29ba10f88819ba9ae2 [term 1 LEADER]: Becoming Leader. State: Replica: 4ebd9481251f4a29ba10f88819ba9ae2, State: Running, Role: LEADER
I20260807 15:11:51.258616 7054 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4ebd9481251f4a29ba10f88819ba9ae2 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:11:51.258759 7062 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4ebd9481251f4a29ba10f88819ba9ae2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ebd9481251f4a29ba10f88819ba9ae2" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 39853 } }
I20260807 15:11:51.259248 7065 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4ebd9481251f4a29ba10f88819ba9ae2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4ebd9481251f4a29ba10f88819ba9ae2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ebd9481251f4a29ba10f88819ba9ae2" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 39853 } } }
I20260807 15:11:51.259358 7065 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4ebd9481251f4a29ba10f88819ba9ae2 [sys.catalog]: This master's current role is: LEADER
I20260807 15:11:51.259672 7066 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4ebd9481251f4a29ba10f88819ba9ae2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4ebd9481251f4a29ba10f88819ba9ae2. Latest consensus state: current_term: 1 leader_uuid: "4ebd9481251f4a29ba10f88819ba9ae2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ebd9481251f4a29ba10f88819ba9ae2" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 39853 } } }
I20260807 15:11:51.259944 7066 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4ebd9481251f4a29ba10f88819ba9ae2 [sys.catalog]: This master's current role is: LEADER
I20260807 15:11:51.260120 7073 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:11:51.260766 7073 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:11:51.262838 7073 catalog_manager.cc:1347] Generated new cluster ID: b1e8937b172c41a9914d5edb2b5e84c5
I20260807 15:11:51.263337 7073 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 07 15:11:51 dist-test-slave-42z9 krb5kdc[6944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115511, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.29.194.62@KRBTEST.COM
I20260807 15:11:51.268811 7061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:51.255195 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43310 (local address 127.29.194.62:39853)
0807 15:11:51.255481 (+ 286us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:51.255492 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:51.255529 (+ 37us) server_negotiation.cc:407] Connection header received
0807 15:11:51.256127 (+ 598us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:51.256139 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:51.256209 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:51.256482 (+ 273us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:11:51.257272 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:51.258664 (+ 1392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:51.260060 (+ 1396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:51.260807 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:51.260931 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:51.264427 (+ 3496us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:11:51.264433 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:11:51.264433 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:11:51.264450 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:11:51.266820 (+ 2370us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:51.267371 (+ 551us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:51.267375 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:51.267375 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:51.267553 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:51.267757 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:51.267758 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:51.267759 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:51.268132 (+ 373us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:11:51.268311 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0807 15:11:51.268388 (+ 77us) server_negotiation.cc:299] Negotiation successful
0807 15:11:51.268573 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":225,"thread_start_us":146,"threads_started":1}
I20260807 15:11:51.273667 7073 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:11:51.274215 7073 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:11:51.279131 7073 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4ebd9481251f4a29ba10f88819ba9ae2: Generated new TSK 0
I20260807 15:11:51.279313 7073 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.29.194.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.29.194.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.29.194.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115480851507-30472-0/krb5kdc/oryx_127.29.194.1.keytab.
Entry for principal oryx/127.29.194.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115480851507-30472-0/krb5kdc/oryx_127.29.194.1.keytab.
I20260807 15:11:51.317504 30472 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7fZSIL/build/release/bin/kudu
/tmp/dist-test-task7fZSIL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115480851507-30472-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115480851507-30472-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115480851507-30472-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115480851507-30472-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.194.1:0
--local_ip_for_outbound_sockets=127.29.194.1
--webserver_interface=127.29.194.1
--webserver_port=0
--tserver_master_addrs=127.29.194.62:39853
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115480851507-30472-0/krb5kdc/oryx_127.29.194.1.keytab
--principal=oryx/127.29.194.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.194.20:45569
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115480851507-30472-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115480851507-30472-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115480851507-30472-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:11:51.410302 7092 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:11:51.410557 7092 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:11:51.410595 7092 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:11:51.412441 7092 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:11:51.412575 7092 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:11:51.412632 7092 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.194.1
I20260807 15:11:51.415333 7092 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.194.20:45569
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115480851507-30472-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115480851507-30472-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115480851507-30472-0/krb5kdc/oryx_127.29.194.1.keytab
--principal=oryx/127.29.194.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.194.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115480851507-30472-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.194.1
--webserver_port=0
--tserver_master_addrs=127.29.194.62:39853
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7092
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.194.1
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115480851507-30472-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:11:51.415656 7092 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:11:51.415915 7092 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:11:51.422173 7105 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:11:51.422160 7102 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:11:51.422148 7100 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:11:51.422683 7092 server_base.cc:1034] running on GCE node
I20260807 15:11:51.423036 7092 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:11:51.423368 7092 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:11:51.424605 7092 hybrid_clock.cc:648] HybridClock initialized: now 1786115511424571 us; error 59 us; skew 500 ppm
Aug 07 15:11:51 dist-test-slave-42z9 krb5kdc[6944](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115511, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.194.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:11:51.427605 7092 init.cc:377] Logged in from keytab as oryx/127.29.194.1@KRBTEST.COM (short username oryx)
I20260807 15:11:51.428122 7092 webserver.cc:460] Webserver started at http://127.29.194.1:40935/ using document root <none> and password file <none>
I20260807 15:11:51.428341 7092 fs_manager.cc:359] Metadata directory not provided
I20260807 15:11:51.428391 7092 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:11:51.428534 7092 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:11:51.429710 7092 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115480851507-30472-0/minicluster-data/ts-0/data/instance:
uuid: "10d07cb0cc8840bea5ce1292f24169f3"
format_stamp: "Formatted at 2026-08-07 15:11:51 on dist-test-slave-42z9"
server_key: "b627f490c1cba251174002f94699f33b"
server_key_iv: "023af06e57c7b6422e51248be14a82b7"
server_key_version: "encryptionkey@0"
I20260807 15:11:51.430073 7092 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115480851507-30472-0/minicluster-data/ts-0/wal/instance:
uuid: "10d07cb0cc8840bea5ce1292f24169f3"
format_stamp: "Formatted at 2026-08-07 15:11:51 on dist-test-slave-42z9"
server_key: "b627f490c1cba251174002f94699f33b"
server_key_iv: "023af06e57c7b6422e51248be14a82b7"
server_key_version: "encryptionkey@0"
I20260807 15:11:51.431538 7092 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:11:51.432446 7116 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:11:51.432662 7092 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:11:51.432760 7092 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115480851507-30472-0/minicluster-data/ts-0/data,/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115480851507-30472-0/minicluster-data/ts-0/wal
uuid: "10d07cb0cc8840bea5ce1292f24169f3"
format_stamp: "Formatted at 2026-08-07 15:11:51 on dist-test-slave-42z9"
server_key: "b627f490c1cba251174002f94699f33b"
server_key_iv: "023af06e57c7b6422e51248be14a82b7"
server_key_version: "encryptionkey@0"
I20260807 15:11:51.432826 7092 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115480851507-30472-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115480851507-30472-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115480851507-30472-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:11:51.444077 7092 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:11:51.446931 7092 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:11:51.447113 7092 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:11:51.447376 7092 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:11:51.447727 7092 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:11:51.447764 7092 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:11:51.447796 7092 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:11:51.447813 7092 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:11:51.454026 7092 rpc_server.cc:307] RPC server started. Bound to: 127.29.194.1:34085
I20260807 15:11:51.454078 7264 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.194.1:34085 every 8 connection(s)
I20260807 15:11:51.454414 7092 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115480851507-30472-0/minicluster-data/ts-0/data/info.pb
Aug 07 15:11:51 dist-test-slave-42z9 krb5kdc[6944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115511, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.194.1@KRBTEST.COM for oryx/127.29.194.62@KRBTEST.COM
I20260807 15:11:51.464208 30472 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7fZSIL/build/release/bin/kudu as pid 7092
I20260807 15:11:51.464293 30472 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115480851507-30472-0/minicluster-data/ts-0/wal/instance
I20260807 15:11:51.464468 30472 external_mini_cluster.cc:1428] Setting key 9c0ddebaebe1887b3d6a28d36cb3d911
I20260807 15:11:51.466056 7061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:51.455196 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.194.1:49835 (local address 127.29.194.62:39853)
0807 15:11:51.455254 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:51.455256 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:51.455519 (+ 263us) server_negotiation.cc:407] Connection header received
0807 15:11:51.456142 (+ 623us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:51.456144 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:51.456155 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:51.456208 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:11:51.457622 (+ 1414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:51.458168 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:51.459534 (+ 1366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:51.459784 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:51.459852 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:51.462507 (+ 2655us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:11:51.462509 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:11:51.462509 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:11:51.462521 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:11:51.463915 (+ 1394us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:51.464444 (+ 529us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:51.464445 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:51.464445 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:51.464536 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:51.464845 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:51.464847 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:51.464847 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:51.465028 (+ 181us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:11:51.465116 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0807 15:11:51.465868 (+ 752us) server_negotiation.cc:299] Negotiation successful
0807 15:11:51.465977 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260807 15:11:51.466534 7268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:51.455160 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:39853 (local address 127.29.194.1:49835)
0807 15:11:51.455331 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0807 15:11:51.455377 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0807 15:11:51.455913 (+ 536us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:11:51.456339 (+ 426us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:11:51.456345 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:11:51.457159 (+ 814us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:11:51.457496 (+ 337us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:51.457505 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:51.458324 (+ 819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:51.458326 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:51.459377 (+ 1051us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:51.459382 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:51.459920 (+ 538us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:51.459921 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:51.460086 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:51.460759 (+ 673us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:11:51.460769 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:11:51.462403 (+ 1634us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:11:51.464043 (+ 1640us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:11:51.464054 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:11:51.464056 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:11:51.464336 (+ 280us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:11:51.464601 (+ 265us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:11:51.464602 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:11:51.464602 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:11:51.464743 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:11:51.465240 (+ 497us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:11:51.465260 (+ 20us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:11:51.465571 (+ 311us) client_negotiation.cc:769] Sending connection context
0807 15:11:51.465794 (+ 223us) client_negotiation.cc:240] Negotiation successful
0807 15:11:51.466081 (+ 287us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":78,"threads_started":1}
I20260807 15:11:51.467267 7266 heartbeater.cc:347] Connected to a master server at 127.29.194.62:39853
I20260807 15:11:51.467404 7266 heartbeater.cc:464] Registering TS with master...
I20260807 15:11:51.467643 7266 heartbeater.cc:511] Master 127.29.194.62:39853 requested a full tablet report, sending...
I20260807 15:11:51.468271 7012 ts_manager.cc:194] Registered new tserver with Master: 10d07cb0cc8840bea5ce1292f24169f3 (127.29.194.1:34085)
I20260807 15:11:51.469167 7012 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.29.194.1@KRBTEST.COM'} at 127.29.194.1:49835
WARNING: no policy specified for oryx/127.29.194.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.29.194.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.29.194.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115480851507-30472-0/krb5kdc/oryx_127.29.194.2.keytab.
Entry for principal oryx/127.29.194.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115480851507-30472-0/krb5kdc/oryx_127.29.194.2.keytab.
I20260807 15:11:51.501327 30472 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7fZSIL/build/release/bin/kudu
/tmp/dist-test-task7fZSIL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115480851507-30472-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115480851507-30472-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115480851507-30472-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115480851507-30472-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.194.2:0
--local_ip_for_outbound_sockets=127.29.194.2
--webserver_interface=127.29.194.2
--webserver_port=0
--tserver_master_addrs=127.29.194.62:39853
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115480851507-30472-0/krb5kdc/oryx_127.29.194.2.keytab
--principal=oryx/127.29.194.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.194.20:45569
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115480851507-30472-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115480851507-30472-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115480851507-30472-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:11:51.591006 7271 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:11:51.591236 7271 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:11:51.591264 7271 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:11:51.593302 7271 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:11:51.593379 7271 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:11:51.593415 7271 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.194.2
I20260807 15:11:51.596192 7271 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.194.20:45569
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115480851507-30472-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115480851507-30472-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115480851507-30472-0/krb5kdc/oryx_127.29.194.2.keytab
--principal=oryx/127.29.194.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.194.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115480851507-30472-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.194.2
--webserver_port=0
--tserver_master_addrs=127.29.194.62:39853
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7271
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.194.2
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115480851507-30472-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:11:51.596498 7271 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:11:51.596774 7271 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:11:51.602837 7288 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:11:51.602850 7283 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:11:51.602864 7285 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:11:51.603111 7271 server_base.cc:1034] running on GCE node
I20260807 15:11:51.603410 7271 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:11:51.603713 7271 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:11:51.604959 7271 hybrid_clock.cc:648] HybridClock initialized: now 1786115511604934 us; error 44 us; skew 500 ppm
Aug 07 15:11:51 dist-test-slave-42z9 krb5kdc[6944](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115511, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.194.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:11:51.608078 7271 init.cc:377] Logged in from keytab as oryx/127.29.194.2@KRBTEST.COM (short username oryx)
I20260807 15:11:51.608654 7271 webserver.cc:460] Webserver started at http://127.29.194.2:44177/ using document root <none> and password file <none>
I20260807 15:11:51.608870 7271 fs_manager.cc:359] Metadata directory not provided
I20260807 15:11:51.608920 7271 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:11:51.609055 7271 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:11:51.610244 7271 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115480851507-30472-0/minicluster-data/ts-1/data/instance:
uuid: "1a18fd2e4123445cad3e1fa79a7fb96c"
format_stamp: "Formatted at 2026-08-07 15:11:51 on dist-test-slave-42z9"
server_key: "1d6dd3b3c75ea5611c99e5705547ca77"
server_key_iv: "29fbe14ca4cdca03d4b260283090b84d"
server_key_version: "encryptionkey@0"
I20260807 15:11:51.610622 7271 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115480851507-30472-0/minicluster-data/ts-1/wal/instance:
uuid: "1a18fd2e4123445cad3e1fa79a7fb96c"
format_stamp: "Formatted at 2026-08-07 15:11:51 on dist-test-slave-42z9"
server_key: "1d6dd3b3c75ea5611c99e5705547ca77"
server_key_iv: "29fbe14ca4cdca03d4b260283090b84d"
server_key_version: "encryptionkey@0"
I20260807 15:11:51.612167 7271 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 15:11:51.613420 7298 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:11:51.613658 7271 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:11:51.613765 7271 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115480851507-30472-0/minicluster-data/ts-1/data,/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115480851507-30472-0/minicluster-data/ts-1/wal
uuid: "1a18fd2e4123445cad3e1fa79a7fb96c"
format_stamp: "Formatted at 2026-08-07 15:11:51 on dist-test-slave-42z9"
server_key: "1d6dd3b3c75ea5611c99e5705547ca77"
server_key_iv: "29fbe14ca4cdca03d4b260283090b84d"
server_key_version: "encryptionkey@0"
I20260807 15:11:51.613839 7271 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115480851507-30472-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115480851507-30472-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115480851507-30472-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:11:51.641644 7271 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:11:51.644248 7271 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:11:51.644407 7271 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:11:51.644693 7271 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:11:51.645123 7271 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:11:51.645183 7271 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:11:51.645217 7271 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:11:51.645236 7271 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:11:51.652468 7271 rpc_server.cc:307] RPC server started. Bound to: 127.29.194.2:32991
I20260807 15:11:51.652577 7450 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.194.2:32991 every 8 connection(s)
I20260807 15:11:51.652879 7271 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115480851507-30472-0/minicluster-data/ts-1/data/info.pb
I20260807 15:11:51.659610 30472 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7fZSIL/build/release/bin/kudu as pid 7271
I20260807 15:11:51.659740 30472 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115480851507-30472-0/minicluster-data/ts-1/wal/instance
I20260807 15:11:51.659982 30472 external_mini_cluster.cc:1428] Setting key 3747f999ed748f4b36b3cf5a7f6de05d
Aug 07 15:11:51 dist-test-slave-42z9 krb5kdc[6944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115511, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.194.2@KRBTEST.COM for oryx/127.29.194.62@KRBTEST.COM
I20260807 15:11:51.663460 7061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:51.653716 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.194.2:49699 (local address 127.29.194.62:39853)
0807 15:11:51.653795 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:51.653798 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:51.653951 (+ 153us) server_negotiation.cc:407] Connection header received
0807 15:11:51.654489 (+ 538us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:51.654491 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:51.654502 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:51.654570 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:11:51.655607 (+ 1037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:51.656142 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:51.657165 (+ 1023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:51.657378 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:51.657480 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:51.660766 (+ 3286us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:11:51.660768 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:11:51.660768 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:11:51.660777 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:11:51.662063 (+ 1286us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:51.662487 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:51.662488 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:51.662489 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:51.662551 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:51.662786 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:51.662787 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:51.662788 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:51.662910 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:11:51.662970 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0807 15:11:51.663263 (+ 293us) server_negotiation.cc:299] Negotiation successful
0807 15:11:51.663365 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260807 15:11:51.663659 7456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:51.653643 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:39853 (local address 127.29.194.2:49699)
0807 15:11:51.653809 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0807 15:11:51.653857 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0807 15:11:51.654376 (+ 519us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:11:51.654633 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:11:51.654636 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:11:51.655278 (+ 642us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:11:51.655492 (+ 214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:51.655497 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:51.656234 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:51.656235 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:51.657065 (+ 830us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:51.657068 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:51.657630 (+ 562us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:51.657633 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:51.657856 (+ 223us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:51.658793 (+ 937us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:11:51.658804 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:11:51.660678 (+ 1874us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:11:51.662151 (+ 1473us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:11:51.662161 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:11:51.662164 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:11:51.662415 (+ 251us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:11:51.662607 (+ 192us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:11:51.662608 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:11:51.662609 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:11:51.662729 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:11:51.662968 (+ 239us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:11:51.662975 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:11:51.663139 (+ 164us) client_negotiation.cc:769] Sending connection context
0807 15:11:51.663237 (+ 98us) client_negotiation.cc:240] Negotiation successful
0807 15:11:51.663422 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":69,"threads_started":1}
I20260807 15:11:51.664113 7453 heartbeater.cc:347] Connected to a master server at 127.29.194.62:39853
I20260807 15:11:51.664218 7453 heartbeater.cc:464] Registering TS with master...
I20260807 15:11:51.664402 7453 heartbeater.cc:511] Master 127.29.194.62:39853 requested a full tablet report, sending...
I20260807 15:11:51.664788 7012 ts_manager.cc:194] Registered new tserver with Master: 1a18fd2e4123445cad3e1fa79a7fb96c (127.29.194.2:32991)
I20260807 15:11:51.665160 7012 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.29.194.2@KRBTEST.COM'} at 127.29.194.2:49699
WARNING: no policy specified for oryx/127.29.194.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.29.194.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.29.194.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115480851507-30472-0/krb5kdc/oryx_127.29.194.3.keytab.
Entry for principal oryx/127.29.194.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115480851507-30472-0/krb5kdc/oryx_127.29.194.3.keytab.
I20260807 15:11:51.692689 30472 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7fZSIL/build/release/bin/kudu
/tmp/dist-test-task7fZSIL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115480851507-30472-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115480851507-30472-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115480851507-30472-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115480851507-30472-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.194.3:0
--local_ip_for_outbound_sockets=127.29.194.3
--webserver_interface=127.29.194.3
--webserver_port=0
--tserver_master_addrs=127.29.194.62:39853
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115480851507-30472-0/krb5kdc/oryx_127.29.194.3.keytab
--principal=oryx/127.29.194.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.194.20:45569
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115480851507-30472-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115480851507-30472-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115480851507-30472-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:11:51.782672 7461 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:11:51.782907 7461 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:11:51.782933 7461 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:11:51.784173 7461 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:11:51.784217 7461 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:11:51.784246 7461 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.194.3
I20260807 15:11:51.785914 7461 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.194.20:45569
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115480851507-30472-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115480851507-30472-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115480851507-30472-0/krb5kdc/oryx_127.29.194.3.keytab
--principal=oryx/127.29.194.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.194.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115480851507-30472-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.194.3
--webserver_port=0
--tserver_master_addrs=127.29.194.62:39853
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7461
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.194.3
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115480851507-30472-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:11:51.786164 7461 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:11:51.786370 7461 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:11:51.791644 7469 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:11:51.791644 7471 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:11:51.791836 7474 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:11:51.792002 7461 server_base.cc:1034] running on GCE node
I20260807 15:11:51.792243 7461 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:11:51.792622 7461 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:11:51.793812 7461 hybrid_clock.cc:648] HybridClock initialized: now 1786115511793790 us; error 43 us; skew 500 ppm
Aug 07 15:11:51 dist-test-slave-42z9 krb5kdc[6944](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115511, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.194.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:11:51.797063 7461 init.cc:377] Logged in from keytab as oryx/127.29.194.3@KRBTEST.COM (short username oryx)
I20260807 15:11:51.797648 7461 webserver.cc:460] Webserver started at http://127.29.194.3:46319/ using document root <none> and password file <none>
I20260807 15:11:51.797884 7461 fs_manager.cc:359] Metadata directory not provided
I20260807 15:11:51.797937 7461 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:11:51.798077 7461 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:11:51.799191 7461 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115480851507-30472-0/minicluster-data/ts-2/data/instance:
uuid: "9d215d5c47fd4619ad846c7151ad85c1"
format_stamp: "Formatted at 2026-08-07 15:11:51 on dist-test-slave-42z9"
server_key: "74ab483c6f40a59242524869d261d3e6"
server_key_iv: "4988e34e950d3782d32a8affa81de027"
server_key_version: "encryptionkey@0"
I20260807 15:11:51.799597 7461 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115480851507-30472-0/minicluster-data/ts-2/wal/instance:
uuid: "9d215d5c47fd4619ad846c7151ad85c1"
format_stamp: "Formatted at 2026-08-07 15:11:51 on dist-test-slave-42z9"
server_key: "74ab483c6f40a59242524869d261d3e6"
server_key_iv: "4988e34e950d3782d32a8affa81de027"
server_key_version: "encryptionkey@0"
I20260807 15:11:51.801164 7461 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260807 15:11:51.802191 7486 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:11:51.802443 7461 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 15:11:51.802556 7461 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115480851507-30472-0/minicluster-data/ts-2/data,/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115480851507-30472-0/minicluster-data/ts-2/wal
uuid: "9d215d5c47fd4619ad846c7151ad85c1"
format_stamp: "Formatted at 2026-08-07 15:11:51 on dist-test-slave-42z9"
server_key: "74ab483c6f40a59242524869d261d3e6"
server_key_iv: "4988e34e950d3782d32a8affa81de027"
server_key_version: "encryptionkey@0"
I20260807 15:11:51.802655 7461 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115480851507-30472-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115480851507-30472-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115480851507-30472-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:11:51.827351 7461 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:11:51.829939 7461 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:11:51.830086 7461 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:11:51.830354 7461 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:11:51.830678 7461 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:11:51.830713 7461 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:11:51.830736 7461 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:11:51.830760 7461 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:11:51.836947 7461 rpc_server.cc:307] RPC server started. Bound to: 127.29.194.3:39783
I20260807 15:11:51.836997 7656 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.194.3:39783 every 8 connection(s)
I20260807 15:11:51.837342 7461 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115480851507-30472-0/minicluster-data/ts-2/data/info.pb
I20260807 15:11:51.840359 30472 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7fZSIL/build/release/bin/kudu as pid 7461
I20260807 15:11:51.840461 30472 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115480851507-30472-0/minicluster-data/ts-2/wal/instance
I20260807 15:11:51.840657 30472 external_mini_cluster.cc:1428] Setting key 5e816216456a8fb868786243f84bf9cc
Aug 07 15:11:51 dist-test-slave-42z9 krb5kdc[6944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115511, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.194.3@KRBTEST.COM for oryx/127.29.194.62@KRBTEST.COM
I20260807 15:11:51.848971 7061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:51.838275 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.194.3:59771 (local address 127.29.194.62:39853)
0807 15:11:51.838375 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:51.838378 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:51.838669 (+ 291us) server_negotiation.cc:407] Connection header received
0807 15:11:51.839274 (+ 605us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:51.839276 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:51.839291 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:51.839356 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:11:51.840391 (+ 1035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:51.841029 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:51.842089 (+ 1060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:51.842289 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:51.842347 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:51.845421 (+ 3074us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:11:51.845424 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:11:51.845424 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:11:51.845434 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:11:51.847396 (+ 1962us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:51.847969 (+ 573us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:51.847971 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:51.847971 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:51.848040 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:51.848278 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:51.848279 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:51.848279 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:51.848410 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:11:51.848471 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0807 15:11:51.848764 (+ 293us) server_negotiation.cc:299] Negotiation successful
0807 15:11:51.848864 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260807 15:11:51.849162 7662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:51.838108 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:39853 (local address 127.29.194.3:59771)
0807 15:11:51.838389 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0807 15:11:51.838479 (+ 90us) client_negotiation.cc:174] Beginning negotiation
0807 15:11:51.839139 (+ 660us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:11:51.839434 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:11:51.839438 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:11:51.840064 (+ 626us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:11:51.840279 (+ 215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:51.840285 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:51.841120 (+ 835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:51.841121 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:51.841996 (+ 875us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:51.841999 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:51.842365 (+ 366us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:51.842366 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:51.842496 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:51.843318 (+ 822us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:11:51.843334 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:11:51.845248 (+ 1914us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:11:51.847546 (+ 2298us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:11:51.847561 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:11:51.847565 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:11:51.847880 (+ 315us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:11:51.848100 (+ 220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:11:51.848101 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:11:51.848101 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:11:51.848219 (+ 118us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:11:51.848471 (+ 252us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:11:51.848479 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:11:51.848633 (+ 154us) client_negotiation.cc:769] Sending connection context
0807 15:11:51.848729 (+ 96us) client_negotiation.cc:240] Negotiation successful
0807 15:11:51.848919 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":81,"threads_started":1}
I20260807 15:11:51.849776 7659 heartbeater.cc:347] Connected to a master server at 127.29.194.62:39853
I20260807 15:11:51.849903 7659 heartbeater.cc:464] Registering TS with master...
I20260807 15:11:51.850131 7659 heartbeater.cc:511] Master 127.29.194.62:39853 requested a full tablet report, sending...
I20260807 15:11:51.850602 7012 ts_manager.cc:194] Registered new tserver with Master: 9d215d5c47fd4619ad846c7151ad85c1 (127.29.194.3:39783)
I20260807 15:11:51.850998 7012 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.29.194.3@KRBTEST.COM'} at 127.29.194.3:59771
I20260807 15:11:51.852655 30472 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.29.194.61@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.29.194.61@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.29.194.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115480851507-30472-0/krb5kdc/oryx_127.29.194.61.keytab.
Entry for principal oryx/127.29.194.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115480851507-30472-0/krb5kdc/oryx_127.29.194.61.keytab.
I20260807 15:11:51.885574 30472 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7fZSIL/build/release/bin/kudu
/tmp/dist-test-task7fZSIL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115480851507-30472-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115480851507-30472-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115480851507-30472-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115480851507-30472-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.194.61:40955
--webserver_interface=127.29.194.61
--webserver_port=0
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115480851507-30472-0/krb5kdc/oryx_127.29.194.61.keytab
--principal=oryx/127.29.194.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.194.20:45569
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.194.62:39853,127.29.194.61:40955
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115480851507-30472-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115480851507-30472-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115480851507-30472-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:11:51.974372 7668 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:11:51.974637 7668 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:11:51.974663 7668 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:11:51.975867 7668 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:11:51.975911 7668 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:11:51.975924 7668 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:11:51.975936 7668 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:11:51.975947 7668 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:11:51.977557 7668 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.194.20:45569
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115480851507-30472-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115480851507-30472-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.29.194.62:39853,127.29.194.61:40955
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115480851507-30472-0/krb5kdc/oryx_127.29.194.61.keytab
--principal=oryx/127.29.194.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.194.61:40955
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115480851507-30472-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.29.194.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7668
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115480851507-30472-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
W20260807 15:11:51.977686 7668 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.29.194.62:39853,127.29.194.61:40955'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260807 15:11:51.977810 7668 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:11:51.978008 7668 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:11:51.983137 7678 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:11:51.983260 7674 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:11:51.983137 7675 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:11:51.983414 7668 server_base.cc:1034] running on GCE node
I20260807 15:11:51.983726 7668 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:11:51.984047 7668 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:11:51.985231 7668 hybrid_clock.cc:648] HybridClock initialized: now 1786115511985223 us; error 39 us; skew 500 ppm
Aug 07 15:11:51 dist-test-slave-42z9 krb5kdc[6944](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115511, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.194.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:11:51.988875 7668 init.cc:377] Logged in from keytab as oryx/127.29.194.61@KRBTEST.COM (short username oryx)
I20260807 15:11:51.989675 7668 webserver.cc:460] Webserver started at http://127.29.194.61:33669/ using document root <none> and password file <none>
I20260807 15:11:51.990003 7668 fs_manager.cc:359] Metadata directory not provided
I20260807 15:11:51.990067 7668 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:11:51.990238 7668 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:11:51.991779 7668 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115480851507-30472-0/minicluster-data/master-1/data/instance:
uuid: "8e78c2a81ab3476b8cbeff1833835aeb"
format_stamp: "Formatted at 2026-08-07 15:11:51 on dist-test-slave-42z9"
server_key: "27737b52fea85fbbe0bc1477630af369"
server_key_iv: "3dbd957e6c85207bbc48410b7933b55b"
server_key_version: "encryptionkey@0"
I20260807 15:11:51.992323 7668 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115480851507-30472-0/minicluster-data/master-1/wal/instance:
uuid: "8e78c2a81ab3476b8cbeff1833835aeb"
format_stamp: "Formatted at 2026-08-07 15:11:51 on dist-test-slave-42z9"
server_key: "27737b52fea85fbbe0bc1477630af369"
server_key_iv: "3dbd957e6c85207bbc48410b7933b55b"
server_key_version: "encryptionkey@0"
I20260807 15:11:51.993824 7668 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.003s
I20260807 15:11:51.994913 7685 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:11:51.995246 7668 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:11:51.995472 7668 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115480851507-30472-0/minicluster-data/master-1/data,/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115480851507-30472-0/minicluster-data/master-1/wal
uuid: "8e78c2a81ab3476b8cbeff1833835aeb"
format_stamp: "Formatted at 2026-08-07 15:11:51 on dist-test-slave-42z9"
server_key: "27737b52fea85fbbe0bc1477630af369"
server_key_iv: "3dbd957e6c85207bbc48410b7933b55b"
server_key_version: "encryptionkey@0"
I20260807 15:11:51.995605 7668 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115480851507-30472-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115480851507-30472-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115480851507-30472-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:11:52.023339 7668 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:11:52.026955 7668 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:11:52.027138 7668 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:11:52.031493 7668 rpc_server.cc:307] RPC server started. Bound to: 127.29.194.61:40955
I20260807 15:11:52.031569 7764 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.194.61:40955 every 8 connection(s)
I20260807 15:11:52.031921 7668 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115480851507-30472-0/minicluster-data/master-1/data/info.pb
I20260807 15:11:52.032475 30472 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7fZSIL/build/release/bin/kudu as pid 7668
I20260807 15:11:52.033114 7767 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:11:52.035672 7767 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.194.62" port: 39853 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 15:11:52 dist-test-slave-42z9 krb5kdc[6944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115511, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.29.194.61@KRBTEST.COM
Aug 07 15:11:52 dist-test-slave-42z9 krb5kdc[6944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115511, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.194.61@KRBTEST.COM for oryx/127.29.194.62@KRBTEST.COM
I20260807 15:11:52.044557 7775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:52.033291 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51630 (local address 127.29.194.61:40955)
0807 15:11:52.033481 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:52.033492 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:52.033520 (+ 28us) server_negotiation.cc:407] Connection header received
0807 15:11:52.033987 (+ 467us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:52.033995 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:52.034053 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:52.034178 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:11:52.034999 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:52.036111 (+ 1112us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:52.037207 (+ 1096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:52.038072 (+ 865us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:52.038214 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:52.041371 (+ 3157us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:11:52.041380 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:11:52.041380 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:11:52.041387 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:11:52.043042 (+ 1655us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:52.043400 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:52.043404 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:52.043404 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:52.043492 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:52.043678 (+ 186us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:52.043679 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:52.043679 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:52.043936 (+ 257us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:11:52.044072 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0807 15:11:52.044225 (+ 153us) server_negotiation.cc:299] Negotiation successful
0807 15:11:52.044358 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":117,"thread_start_us":69,"threads_started":1}
I20260807 15:11:52.046108 7061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:52.036509 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43324 (local address 127.29.194.62:39853)
0807 15:11:52.036575 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:52.036577 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:52.036848 (+ 271us) server_negotiation.cc:407] Connection header received
0807 15:11:52.036907 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:52.036909 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:52.036927 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:52.037029 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:11:52.037613 (+ 584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:52.038498 (+ 885us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:52.039758 (+ 1260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:52.040103 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:52.040230 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:52.042607 (+ 2377us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:11:52.042609 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:11:52.042609 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:11:52.042620 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:11:52.044610 (+ 1990us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:52.045068 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:52.045069 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:52.045069 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:52.045151 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:52.045404 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:52.045405 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:52.045406 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:52.045690 (+ 284us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:11:52.045756 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0807 15:11:52.045954 (+ 198us) server_negotiation.cc:299] Negotiation successful
0807 15:11:52.046041 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260807 15:11:52.046125 7777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:52.036413 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:39853 (local address 127.0.0.1:43324)
0807 15:11:52.036649 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0807 15:11:52.036697 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0807 15:11:52.036780 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:11:52.037070 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:11:52.037074 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:11:52.037398 (+ 324us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:11:52.037518 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:52.037523 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:52.038651 (+ 1128us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:52.038653 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:52.039618 (+ 965us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:52.039622 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:52.040265 (+ 643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:52.040266 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:52.040385 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:52.040970 (+ 585us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:11:52.040979 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:11:52.042444 (+ 1465us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:11:52.044738 (+ 2294us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:11:52.044747 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:11:52.044749 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:11:52.044980 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:11:52.045218 (+ 238us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:11:52.045219 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:11:52.045219 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:11:52.045297 (+ 78us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:11:52.045776 (+ 479us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:11:52.045783 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:11:52.045833 (+ 50us) client_negotiation.cc:769] Sending connection context
0807 15:11:52.045934 (+ 101us) client_negotiation.cc:240] Negotiation successful
0807 15:11:52.046040 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":92,"threads_started":1}
I20260807 15:11:52.046649 7767 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.194.61" port: 40955 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 15:11:52 dist-test-slave-42z9 krb5kdc[6944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115511, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.194.61@KRBTEST.COM for oryx/127.29.194.61@KRBTEST.COM
I20260807 15:11:52.054683 7777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:52.046923 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.61:40955 (local address 127.0.0.1:51638)
0807 15:11:52.046970 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0807 15:11:52.046981 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 15:11:52.047018 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:11:52.047110 (+ 92us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:11:52.047111 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:11:52.047226 (+ 115us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:11:52.047281 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:52.047283 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:52.048210 (+ 927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:52.048211 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:52.048751 (+ 540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:52.048753 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:52.049269 (+ 516us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:52.049270 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:52.049348 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:52.049962 (+ 614us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:11:52.049967 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:11:52.051345 (+ 1378us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:11:52.053023 (+ 1678us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:11:52.053025 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:11:52.053025 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:11:52.053197 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:11:52.053522 (+ 325us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:11:52.053523 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:11:52.053524 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:11:52.053625 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:11:52.054264 (+ 639us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:11:52.054265 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:11:52.054327 (+ 62us) client_negotiation.cc:769] Sending connection context
0807 15:11:52.054406 (+ 79us) client_negotiation.cc:240] Negotiation successful
0807 15:11:52.054518 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29}
I20260807 15:11:52.054893 7775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:52.046953 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51638 (local address 127.29.194.61:40955)
0807 15:11:52.047001 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:52.047002 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:52.047011 (+ 9us) server_negotiation.cc:407] Connection header received
0807 15:11:52.047058 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:52.047058 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:52.047066 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:52.047106 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:11:52.047433 (+ 327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:52.048021 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:52.048874 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:52.049128 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:52.049203 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:52.051499 (+ 2296us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:11:52.051502 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:11:52.051502 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:11:52.051509 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:11:52.052906 (+ 1397us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:52.053328 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:52.053329 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:52.053329 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:52.053379 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:52.053855 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:52.053857 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:52.053858 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:52.054071 (+ 213us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:11:52.054170 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0807 15:11:52.054496 (+ 326us) server_negotiation.cc:299] Negotiation successful
0807 15:11:52.054625 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260807 15:11:52.056945 7767 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8e78c2a81ab3476b8cbeff1833835aeb: Bootstrap starting.
I20260807 15:11:52.058243 7767 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8e78c2a81ab3476b8cbeff1833835aeb: Neither blocks nor log segments found. Creating new log.
I20260807 15:11:52.058590 7767 log.cc:824] T 00000000000000000000000000000000 P 8e78c2a81ab3476b8cbeff1833835aeb: Log is configured to *not* fsync() on all Append() calls
I20260807 15:11:52.059863 7767 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8e78c2a81ab3476b8cbeff1833835aeb: No bootstrap required, opened a new log
I20260807 15:11:52.061841 7767 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8e78c2a81ab3476b8cbeff1833835aeb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ebd9481251f4a29ba10f88819ba9ae2" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 39853 } } peers { permanent_uuid: "8e78c2a81ab3476b8cbeff1833835aeb" member_type: VOTER last_known_addr { host: "127.29.194.61" port: 40955 } }
I20260807 15:11:52.062059 7767 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8e78c2a81ab3476b8cbeff1833835aeb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:11:52.062101 7767 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8e78c2a81ab3476b8cbeff1833835aeb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8e78c2a81ab3476b8cbeff1833835aeb, State: Initialized, Role: FOLLOWER
I20260807 15:11:52.062255 7767 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8e78c2a81ab3476b8cbeff1833835aeb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ebd9481251f4a29ba10f88819ba9ae2" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 39853 } } peers { permanent_uuid: "8e78c2a81ab3476b8cbeff1833835aeb" member_type: VOTER last_known_addr { host: "127.29.194.61" port: 40955 } }
I20260807 15:11:52.062657 7767 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8e78c2a81ab3476b8cbeff1833835aeb [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:11:52.062693 7779 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8e78c2a81ab3476b8cbeff1833835aeb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ebd9481251f4a29ba10f88819ba9ae2" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 39853 } } peers { permanent_uuid: "8e78c2a81ab3476b8cbeff1833835aeb" member_type: VOTER last_known_addr { host: "127.29.194.61" port: 40955 } } }
I20260807 15:11:52.062856 7779 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8e78c2a81ab3476b8cbeff1833835aeb [sys.catalog]: This master's current role is: FOLLOWER
W20260807 15:11:52.065194 7791 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 8e78c2a81ab3476b8cbeff1833835aeb: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 15:11:52.065267 7791 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 15:11:52.067400 7668 master_runner.cc:428] Detected that this master 8e78c2a81ab3476b8cbeff1833835aeb is joining an existing cluster
I20260807 15:11:52.067484 7668 master_runner.cc:432] Initiating AddMaster RPC to add 127.29.194.61:40955
I20260807 15:11:52.076012 7061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:52.068766 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43334 (local address 127.29.194.62:39853)
0807 15:11:52.068821 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:52.068823 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:52.069387 (+ 564us) server_negotiation.cc:407] Connection header received
0807 15:11:52.069525 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:52.069526 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:52.069543 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:52.069588 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:11:52.069970 (+ 382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:52.070722 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:52.071563 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:52.071781 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:52.071841 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:52.073034 (+ 1193us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:11:52.073036 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:11:52.073036 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:11:52.073044 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:11:52.074695 (+ 1651us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:52.075195 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:52.075197 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:52.075197 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:52.075263 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:52.075424 (+ 161us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:52.075425 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:52.075425 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:52.075550 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:11:52.075603 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0807 15:11:52.075798 (+ 195us) server_negotiation.cc:299] Negotiation successful
0807 15:11:52.075914 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260807 15:11:52.076152 7799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:52.068701 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.61:40955 (local address 127.0.0.1:51646)
0807 15:11:52.069012 (+ 311us) negotiation.cc:107] Waiting for socket to connect
0807 15:11:52.069024 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 15:11:52.069086 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:11:52.069478 (+ 392us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:11:52.069479 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:11:52.069757 (+ 278us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:11:52.069845 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:52.069848 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:52.070801 (+ 953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:52.070802 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:52.071692 (+ 890us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:52.071695 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:52.072052 (+ 357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:52.072053 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:52.072122 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:52.072519 (+ 397us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:11:52.072525 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:11:52.072914 (+ 389us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:11:52.074916 (+ 2002us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:11:52.074917 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:11:52.074917 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:11:52.075133 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:11:52.075457 (+ 324us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:11:52.075458 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:11:52.075458 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:11:52.075536 (+ 78us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:11:52.075802 (+ 266us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:11:52.075803 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:11:52.075852 (+ 49us) client_negotiation.cc:769] Sending connection context
0807 15:11:52.075919 (+ 67us) client_negotiation.cc:240] Negotiation successful
0807 15:11:52.076044 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":282,"thread_start_us":546,"threads_started":1}
I20260807 15:11:52.076565 7775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:52.068807 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51646 (local address 127.29.194.61:40955)
0807 15:11:52.069317 (+ 510us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:52.069318 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:52.069328 (+ 10us) server_negotiation.cc:407] Connection header received
0807 15:11:52.069350 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:52.069351 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:52.069362 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:52.069415 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:11:52.069951 (+ 536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:52.070712 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:52.071771 (+ 1059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:52.071984 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:52.072047 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:52.072993 (+ 946us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:11:52.072995 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:11:52.072995 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:11:52.073002 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:11:52.074854 (+ 1852us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:52.075213 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:52.075214 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:52.075215 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:52.075276 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:52.075607 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:52.075608 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:52.075609 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:52.075742 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:11:52.075805 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0807 15:11:52.075952 (+ 147us) server_negotiation.cc:299] Negotiation successful
0807 15:11:52.076116 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":84,"server-negotiator.queue_time_us":488}
I20260807 15:11:52.076581 7800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:52.068701 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:39853 (local address 127.0.0.1:43334)
0807 15:11:52.069321 (+ 620us) negotiation.cc:107] Waiting for socket to connect
0807 15:11:52.069331 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 15:11:52.069410 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:11:52.069599 (+ 189us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:11:52.069600 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:11:52.069807 (+ 207us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:11:52.069880 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:52.069882 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:52.070808 (+ 926us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:52.070808 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:52.071474 (+ 666us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:52.071476 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:52.071860 (+ 384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:52.071861 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:52.071928 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:52.072402 (+ 474us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:11:52.072408 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:11:52.072913 (+ 505us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:11:52.074809 (+ 1896us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:11:52.074810 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:11:52.074810 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:11:52.075110 (+ 300us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:11:52.075321 (+ 211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:11:52.075322 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:11:52.075323 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:11:52.075372 (+ 49us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:11:52.075629 (+ 257us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:11:52.075630 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:11:52.075679 (+ 49us) client_negotiation.cc:769] Sending connection context
0807 15:11:52.076346 (+ 667us) client_negotiation.cc:240] Negotiation successful
0807 15:11:52.076457 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":606,"mutex_wait_us":195,"thread_start_us":110,"threads_started":1}
Aug 07 15:11:52 dist-test-slave-42z9 krb5kdc[6944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115511, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.194.62@KRBTEST.COM for oryx/127.29.194.61@KRBTEST.COM
I20260807 15:11:52.086508 7801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:52.079105 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.61:40955 (local address 127.0.0.1:51648)
0807 15:11:52.079257 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0807 15:11:52.079296 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0807 15:11:52.079386 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:11:52.079586 (+ 200us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:11:52.079591 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:11:52.079911 (+ 320us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:11:52.079981 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:52.079983 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:52.080609 (+ 626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:52.080610 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:52.081223 (+ 613us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:52.081225 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:52.081622 (+ 397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:52.081623 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:52.081709 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:52.082207 (+ 498us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:11:52.082212 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:11:52.083844 (+ 1632us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:11:52.085341 (+ 1497us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:11:52.085352 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:11:52.085355 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:11:52.085647 (+ 292us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:11:52.085829 (+ 182us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:11:52.085830 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:11:52.085830 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:11:52.085894 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:11:52.086122 (+ 228us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:11:52.086131 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:11:52.086194 (+ 63us) client_negotiation.cc:769] Sending connection context
0807 15:11:52.086271 (+ 77us) client_negotiation.cc:240] Negotiation successful
0807 15:11:52.086409 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":91,"threads_started":1}
I20260807 15:11:52.086500 7775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:52.079156 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51648 (local address 127.29.194.61:40955)
0807 15:11:52.079196 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:52.079198 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:52.079370 (+ 172us) server_negotiation.cc:407] Connection header received
0807 15:11:52.079467 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:52.079468 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:52.079479 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:52.079573 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:11:52.080061 (+ 488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:52.080534 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:52.081309 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:52.081548 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:52.081603 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:52.083942 (+ 2339us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:11:52.083943 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:11:52.083944 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:11:52.083960 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:11:52.085248 (+ 1288us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:52.085717 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:52.085718 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:52.085719 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:52.085774 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:52.085944 (+ 170us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:52.085945 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:52.085945 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:52.086065 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:11:52.086123 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0807 15:11:52.086302 (+ 179us) server_negotiation.cc:299] Negotiation successful
0807 15:11:52.086386 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260807 15:11:52.087287 7012 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.29.194.61:40955: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "8e78c2a81ab3476b8cbeff1833835aeb"
member_type: NON_VOTER
last_known_addr {
host: "127.29.194.61"
port: 40955
}
attrs {
promote: true
}
}
dest_uuid: "4ebd9481251f4a29ba10f88819ba9ae2"
cas_config_opid_index: -1
I20260807 15:11:52.087688 7012 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4ebd9481251f4a29ba10f88819ba9ae2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "4ebd9481251f4a29ba10f88819ba9ae2" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 39853 } } peers { permanent_uuid: "8e78c2a81ab3476b8cbeff1833835aeb" member_type: NON_VOTER last_known_addr { host: "127.29.194.61" port: 40955 } attrs { promote: true } }
I20260807 15:11:52.088274 7803 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 4ebd9481251f4a29ba10f88819ba9ae2 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 8e78c2a81ab3476b8cbeff1833835aeb (127.29.194.61) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "4ebd9481251f4a29ba10f88819ba9ae2" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 39853 } } peers { permanent_uuid: "8e78c2a81ab3476b8cbeff1833835aeb" member_type: NON_VOTER last_known_addr { host: "127.29.194.61" port: 40955 } attrs { promote: true } } }
I20260807 15:11:52.088752 7725 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8e78c2a81ab3476b8cbeff1833835aeb [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:11:52.089555 7803 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.29.194.61:40955
I20260807 15:11:52.089850 7725 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 8e78c2a81ab3476b8cbeff1833835aeb [term 1 FOLLOWER]: Refusing update from remote peer 4ebd9481251f4a29ba10f88819ba9ae2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260807 15:11:52.089986 7668 master.cc:537] Master@127.29.194.61:40955 shutting down...
I20260807 15:11:52.090214 7803 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 4ebd9481251f4a29ba10f88819ba9ae2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e78c2a81ab3476b8cbeff1833835aeb" member_type: NON_VOTER last_known_addr { host: "127.29.194.61" port: 40955 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:11:52.090544 7802 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4ebd9481251f4a29ba10f88819ba9ae2 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "4ebd9481251f4a29ba10f88819ba9ae2" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "4ebd9481251f4a29ba10f88819ba9ae2" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 39853 } } peers { permanent_uuid: "8e78c2a81ab3476b8cbeff1833835aeb" member_type: NON_VOTER last_known_addr { host: "127.29.194.61" port: 40955 } attrs { promote: true } } }
I20260807 15:11:52.090734 7802 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4ebd9481251f4a29ba10f88819ba9ae2 [sys.catalog]: This master's current role is: LEADER
W20260807 15:11:52.090962 6987 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
W20260807 15:11:52.091382 6987 consensus_peers.cc:487] T 00000000000000000000000000000000 P 4ebd9481251f4a29ba10f88819ba9ae2 -> Peer 8e78c2a81ab3476b8cbeff1833835aeb (127.29.194.61:40955): Couldn't send request to peer 8e78c2a81ab3476b8cbeff1833835aeb. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260807 15:11:52.092831 7668 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 8e78c2a81ab3476b8cbeff1833835aeb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 15:11:52.092938 7668 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 8e78c2a81ab3476b8cbeff1833835aeb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:11:52.092995 7668 tablet_replica.cc:330] T 00000000000000000000000000000000 P 8e78c2a81ab3476b8cbeff1833835aeb: stopping tablet replica
I20260807 15:11:52.106030 7668 master.cc:552] Master@127.29.194.61:40955 shutdown complete.
I20260807 15:11:52.106091 7668 master_runner.cc:305] Clearing existing system tablet
I20260807 15:11:52.106218 7668 fs_manager.cc:359] Metadata directory not provided
I20260807 15:11:52.106263 7668 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:11:52.106895 7668 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.001s sys 0.001s
I20260807 15:11:52.107398 7808 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:11:52.107568 7668 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115480851507-30472-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:11:52.107625 7668 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260807 15:11:52.107647 7668 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115480851507-30472-0/minicluster-data/master-1/data,/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115480851507-30472-0/minicluster-data/master-1/wal
uuid: "8e78c2a81ab3476b8cbeff1833835aeb"
format_stamp: "Formatted at 2026-08-07 15:11:51 on dist-test-slave-42z9"
server_key: "27737b52fea85fbbe0bc1477630af369"
server_key_iv: "3dbd957e6c85207bbc48410b7933b55b"
server_key_version: "encryptionkey@0"
I20260807 15:11:52.108415 7668 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 8e78c2a81ab3476b8cbeff1833835aeb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:11:52.110265 7668 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 8e78c2a81ab3476b8cbeff1833835aeb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260807 15:11:52.110335 7668 log.cc:1196] T 00000000000000000000000000000000 P 8e78c2a81ab3476b8cbeff1833835aeb: Deleting WAL directory at /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115480851507-30472-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260807 15:11:52.110531 7668 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 8e78c2a81ab3476b8cbeff1833835aeb: Deleting consensus metadata
I20260807 15:11:52.110706 7668 master_runner.cc:315] Copying system tablet from 127.29.194.62:39853
I20260807 15:11:52.111299 7668 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 8e78c2a81ab3476b8cbeff1833835aeb: tablet copy: Beginning tablet copy session from remote peer at address 127.29.194.62:39853
I20260807 15:11:52.118649 7061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:52.111794 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43344 (local address 127.29.194.62:39853)
0807 15:11:52.111846 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:52.111848 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:52.111898 (+ 50us) server_negotiation.cc:407] Connection header received
0807 15:11:52.111980 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:52.111980 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:52.111990 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:52.112043 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:11:52.112448 (+ 405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:52.112962 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:52.114122 (+ 1160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:52.114320 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:52.114385 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:52.115617 (+ 1232us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:11:52.115618 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:11:52.115619 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:11:52.115626 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:11:52.117377 (+ 1751us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:52.117863 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:52.117865 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:52.117866 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:52.117933 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:52.118155 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:52.118156 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:52.118156 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:52.118283 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:11:52.118339 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0807 15:11:52.118487 (+ 148us) server_negotiation.cc:299] Negotiation successful
0807 15:11:52.118577 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260807 15:11:52.118712 7817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:52.111692 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:39853 (local address 127.0.0.1:43344)
0807 15:11:52.111815 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0807 15:11:52.111827 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 15:11:52.111915 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:11:52.112054 (+ 139us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:11:52.112055 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:11:52.112272 (+ 217us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:11:52.112343 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:52.112345 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:52.113044 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:52.113045 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:52.113981 (+ 936us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:52.113984 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:52.114399 (+ 415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:52.114400 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:52.114473 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:52.115073 (+ 600us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:11:52.115078 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:11:52.115502 (+ 424us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:11:52.117509 (+ 2007us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:11:52.117510 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:11:52.117511 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:11:52.117751 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:11:52.117992 (+ 241us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:11:52.117993 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:11:52.117994 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:11:52.118064 (+ 70us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:11:52.118341 (+ 277us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:11:52.118342 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:11:52.118397 (+ 55us) client_negotiation.cc:769] Sending connection context
0807 15:11:52.118470 (+ 73us) client_negotiation.cc:240] Negotiation successful
0807 15:11:52.118591 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":94,"thread_start_us":65,"threads_started":1}
I20260807 15:11:52.119263 7035 tablet_copy_service.cc:140] P 4ebd9481251f4a29ba10f88819ba9ae2: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 8e78c2a81ab3476b8cbeff1833835aeb ({username='oryx', principal='oryx/127.29.194.61@KRBTEST.COM'} at 127.0.0.1:43344)
I20260807 15:11:52.119417 7035 tablet_copy_service.cc:161] P 4ebd9481251f4a29ba10f88819ba9ae2: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 8e78c2a81ab3476b8cbeff1833835aeb at {username='oryx', principal='oryx/127.29.194.61@KRBTEST.COM'} at 127.0.0.1:43344: session id = 8e78c2a81ab3476b8cbeff1833835aeb-00000000000000000000000000000000
I20260807 15:11:52.120168 7035 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 4ebd9481251f4a29ba10f88819ba9ae2: Tablet Copy: opened 0 blocks and 1 log segments
I20260807 15:11:52.120911 7668 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260807 15:11:52.122231 7668 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 8e78c2a81ab3476b8cbeff1833835aeb: tablet copy: Starting download of 0 data blocks...
I20260807 15:11:52.122372 7668 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 8e78c2a81ab3476b8cbeff1833835aeb: tablet copy: Starting download of 1 WAL segments...
I20260807 15:11:52.123700 7668 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 8e78c2a81ab3476b8cbeff1833835aeb: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260807 15:11:52.124902 7035 tablet_copy_service.cc:342] P 4ebd9481251f4a29ba10f88819ba9ae2: Request end of tablet copy session 8e78c2a81ab3476b8cbeff1833835aeb-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.29.194.61@KRBTEST.COM'} at 127.0.0.1:43344
I20260807 15:11:52.124989 7035 tablet_copy_service.cc:433] P 4ebd9481251f4a29ba10f88819ba9ae2: ending tablet copy session 8e78c2a81ab3476b8cbeff1833835aeb-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 8e78c2a81ab3476b8cbeff1833835aeb
I20260807 15:11:52.128459 7668 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:11:52.129887 7821 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:11:52.130125 7818 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:11:52.130158 7668 server_base.cc:1034] running on GCE node
W20260807 15:11:52.130381 7819 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:11:52.130514 7668 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:11:52.130666 7668 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:11:52.131862 7668 hybrid_clock.cc:648] HybridClock initialized: now 1786115512131821 us; error 57 us; skew 500 ppm
Aug 07 15:11:52 dist-test-slave-42z9 krb5kdc[6944](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115512, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.194.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:11:52.133493 7668 init.cc:377] Logged in from keytab as oryx/127.29.194.61@KRBTEST.COM (short username oryx)
I20260807 15:11:52.133837 7668 webserver.cc:460] Webserver started at http://127.29.194.61:46163/ using document root <none> and password file <none>
I20260807 15:11:52.133970 7668 fs_manager.cc:359] Metadata directory not provided
I20260807 15:11:52.134011 7668 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:11:52.134913 7668 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:11:52.135766 7834 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:11:52.135962 7668 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:11:52.136008 7668 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115480851507-30472-0/minicluster-data/master-1/data,/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115480851507-30472-0/minicluster-data/master-1/wal
uuid: "8e78c2a81ab3476b8cbeff1833835aeb"
format_stamp: "Formatted at 2026-08-07 15:11:51 on dist-test-slave-42z9"
server_key: "27737b52fea85fbbe0bc1477630af369"
server_key_iv: "3dbd957e6c85207bbc48410b7933b55b"
server_key_version: "encryptionkey@0"
I20260807 15:11:52.136092 7668 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115480851507-30472-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115480851507-30472-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115480851507-30472-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:11:52.149063 7668 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:11:52.151177 7668 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:11:52.155112 7668 rpc_server.cc:307] RPC server started. Bound to: 127.29.194.61:40955
I20260807 15:11:52.155464 7909 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.194.61:40955 every 8 connection(s)
I20260807 15:11:52.155520 7668 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115480851507-30472-0/minicluster-data/master-1/data/info.pb
I20260807 15:11:52.156333 7910 sys_catalog.cc:263] Verifying existing consensus state
I20260807 15:11:52.156783 7910 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8e78c2a81ab3476b8cbeff1833835aeb: Bootstrap starting.
I20260807 15:11:52.160192 7910 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8e78c2a81ab3476b8cbeff1833835aeb: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 15:11:52.160437 7910 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8e78c2a81ab3476b8cbeff1833835aeb: Bootstrap complete.
I20260807 15:11:52.160705 7910 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8e78c2a81ab3476b8cbeff1833835aeb [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "4ebd9481251f4a29ba10f88819ba9ae2" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 39853 } } peers { permanent_uuid: "8e78c2a81ab3476b8cbeff1833835aeb" member_type: NON_VOTER last_known_addr { host: "127.29.194.61" port: 40955 } attrs { promote: true } }
I20260807 15:11:52.160796 7910 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8e78c2a81ab3476b8cbeff1833835aeb [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 8e78c2a81ab3476b8cbeff1833835aeb, State: Initialized, Role: LEARNER
I20260807 15:11:52.160825 7910 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8e78c2a81ab3476b8cbeff1833835aeb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "4ebd9481251f4a29ba10f88819ba9ae2" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 39853 } } peers { permanent_uuid: "8e78c2a81ab3476b8cbeff1833835aeb" member_type: NON_VOTER last_known_addr { host: "127.29.194.61" port: 40955 } attrs { promote: true } }
I20260807 15:11:52.161018 7910 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8e78c2a81ab3476b8cbeff1833835aeb [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:11:52.161119 7914 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8e78c2a81ab3476b8cbeff1833835aeb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "4ebd9481251f4a29ba10f88819ba9ae2" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 39853 } } peers { permanent_uuid: "8e78c2a81ab3476b8cbeff1833835aeb" member_type: NON_VOTER last_known_addr { host: "127.29.194.61" port: 40955 } attrs { promote: true } } }
I20260807 15:11:52.161305 7914 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8e78c2a81ab3476b8cbeff1833835aeb [sys.catalog]: This master's current role is: LEARNER
I20260807 15:11:52.162580 7928 catalog_manager.cc:1259] Loaded cluster ID: b1e8937b172c41a9914d5edb2b5e84c5
I20260807 15:11:52.162648 7928 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 8e78c2a81ab3476b8cbeff1833835aeb: loading cluster ID for follower catalog manager: success
I20260807 15:11:52.163851 7928 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 8e78c2a81ab3476b8cbeff1833835aeb: acquiring CA information for follower catalog manager: success
I20260807 15:11:52.164309 7928 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 8e78c2a81ab3476b8cbeff1833835aeb: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 15:11:52.205641 7929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:52.197377 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51676 (local address 127.29.194.61:40955)
0807 15:11:52.197589 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:52.197591 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:52.197598 (+ 7us) server_negotiation.cc:407] Connection header received
0807 15:11:52.197631 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:52.197632 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:52.197645 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:52.197716 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:11:52.198432 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:52.199294 (+ 862us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:52.200097 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:52.200375 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:52.200443 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:52.202093 (+ 1650us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:11:52.202096 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:11:52.202096 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:11:52.202106 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:11:52.204076 (+ 1970us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:52.204746 (+ 670us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:52.204748 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:52.204749 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:52.204821 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:52.205015 (+ 194us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:52.205016 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:52.205016 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:52.205146 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:11:52.205294 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0807 15:11:52.205369 (+ 75us) server_negotiation.cc:299] Negotiation successful
0807 15:11:52.205508 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":187,"thread_start_us":142,"threads_started":1}
WARNING: no policy specified for oryx/127.29.194.60@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.29.194.60@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.29.194.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115480851507-30472-0/krb5kdc/oryx_127.29.194.60.keytab.
Entry for principal oryx/127.29.194.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115480851507-30472-0/krb5kdc/oryx_127.29.194.60.keytab.
I20260807 15:11:52.242509 30472 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7fZSIL/build/release/bin/kudu
/tmp/dist-test-task7fZSIL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115480851507-30472-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115480851507-30472-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115480851507-30472-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115480851507-30472-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.194.60:44209
--webserver_interface=127.29.194.60
--webserver_port=0
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115480851507-30472-0/krb5kdc/oryx_127.29.194.60.keytab
--principal=oryx/127.29.194.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.194.20:45569
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.194.62:39853,127.29.194.61:40955,127.29.194.60:44209
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115480851507-30472-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115480851507-30472-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115480851507-30472-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:11:52.331853 7936 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:11:52.332086 7936 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:11:52.332120 7936 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:11:52.333312 7936 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:11:52.333359 7936 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:11:52.333377 7936 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:11:52.333393 7936 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:11:52.333410 7936 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:11:52.335088 7936 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.194.20:45569
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115480851507-30472-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115480851507-30472-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.29.194.62:39853,127.29.194.61:40955,127.29.194.60:44209
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115480851507-30472-0/krb5kdc/oryx_127.29.194.60.keytab
--principal=oryx/127.29.194.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.194.60:44209
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115480851507-30472-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.29.194.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7936
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115480851507-30472-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:11:52.335348 7936 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:11:52.335616 7936 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:11:52.340850 7950 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:11:52.340911 7943 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:11:52.340883 7942 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:11:52.341116 7936 server_base.cc:1034] running on GCE node
I20260807 15:11:52.341365 7936 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:11:52.341734 7936 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:11:52.342928 7936 hybrid_clock.cc:648] HybridClock initialized: now 1786115512342895 us; error 63 us; skew 500 ppm
Aug 07 15:11:52 dist-test-slave-42z9 krb5kdc[6944](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115512, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.194.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:11:52.346174 7936 init.cc:377] Logged in from keytab as oryx/127.29.194.60@KRBTEST.COM (short username oryx)
I20260807 15:11:52.346663 7936 webserver.cc:460] Webserver started at http://127.29.194.60:43509/ using document root <none> and password file <none>
I20260807 15:11:52.346868 7936 fs_manager.cc:359] Metadata directory not provided
I20260807 15:11:52.346913 7936 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:11:52.347030 7936 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:11:52.348099 7936 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115480851507-30472-0/minicluster-data/master-2/data/instance:
uuid: "e5603c6f9ea84f1a9510499da5950324"
format_stamp: "Formatted at 2026-08-07 15:11:52 on dist-test-slave-42z9"
server_key: "eb40f07c6ebf8c88cb03c225267bfdc8"
server_key_iv: "cbb7178be5d1a530b9f26031cd9d71ea"
server_key_version: "encryptionkey@0"
I20260807 15:11:52.348439 7936 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115480851507-30472-0/minicluster-data/master-2/wal/instance:
uuid: "e5603c6f9ea84f1a9510499da5950324"
format_stamp: "Formatted at 2026-08-07 15:11:52 on dist-test-slave-42z9"
server_key: "eb40f07c6ebf8c88cb03c225267bfdc8"
server_key_iv: "cbb7178be5d1a530b9f26031cd9d71ea"
server_key_version: "encryptionkey@0"
I20260807 15:11:52.349846 7936 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260807 15:11:52.350770 7958 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:11:52.350961 7936 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:11:52.351058 7936 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115480851507-30472-0/minicluster-data/master-2/data,/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115480851507-30472-0/minicluster-data/master-2/wal
uuid: "e5603c6f9ea84f1a9510499da5950324"
format_stamp: "Formatted at 2026-08-07 15:11:52 on dist-test-slave-42z9"
server_key: "eb40f07c6ebf8c88cb03c225267bfdc8"
server_key_iv: "cbb7178be5d1a530b9f26031cd9d71ea"
server_key_version: "encryptionkey@0"
I20260807 15:11:52.351133 7936 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115480851507-30472-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115480851507-30472-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115480851507-30472-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:11:52.363539 7936 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:11:52.365870 7936 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:11:52.366019 7936 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:11:52.370596 7936 rpc_server.cc:307] RPC server started. Bound to: 127.29.194.60:44209
I20260807 15:11:52.370630 8030 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.194.60:44209 every 8 connection(s)
I20260807 15:11:52.370980 7936 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115480851507-30472-0/minicluster-data/master-2/data/info.pb
I20260807 15:11:52.372078 8031 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:11:52.373732 8031 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.194.62" port: 39853 } has no permanent_uuid. Determining permanent_uuid...
I20260807 15:11:52.380612 30472 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7fZSIL/build/release/bin/kudu as pid 7936
Aug 07 15:11:52 dist-test-slave-42z9 krb5kdc[6944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115512, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.194.60@KRBTEST.COM for oryx/127.29.194.62@KRBTEST.COM
I20260807 15:11:52.385654 7061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:52.374636 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43354 (local address 127.29.194.62:39853)
0807 15:11:52.374706 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:52.374708 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:52.374832 (+ 124us) server_negotiation.cc:407] Connection header received
0807 15:11:52.375381 (+ 549us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:52.375383 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:52.375394 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:52.375454 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:11:52.376225 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:52.376852 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:52.378049 (+ 1197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:52.378288 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:52.378365 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:52.381701 (+ 3336us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:11:52.381703 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:11:52.381704 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:11:52.381713 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:11:52.383826 (+ 2113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:52.384354 (+ 528us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:52.384355 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:52.384356 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:52.384436 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:52.384813 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:52.384814 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:52.384815 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:52.384956 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:11:52.385052 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0807 15:11:52.385394 (+ 342us) server_negotiation.cc:299] Negotiation successful
0807 15:11:52.385552 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260807 15:11:52.385790 8034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:52.374487 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:39853 (local address 127.0.0.1:43354)
0807 15:11:52.374718 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0807 15:11:52.374756 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0807 15:11:52.375241 (+ 485us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:11:52.375563 (+ 322us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:11:52.375567 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:11:52.375835 (+ 268us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:11:52.376044 (+ 209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:52.376048 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:52.377014 (+ 966us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:52.377015 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:52.377918 (+ 903us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:52.377922 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:52.378471 (+ 549us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:52.378472 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:52.378661 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:52.379354 (+ 693us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:11:52.379366 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:11:52.381553 (+ 2187us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:11:52.383938 (+ 2385us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:11:52.383945 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:11:52.383948 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:11:52.384267 (+ 319us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:11:52.384530 (+ 263us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:11:52.384531 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:11:52.384531 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:11:52.384696 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:11:52.385062 (+ 366us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:11:52.385067 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:11:52.385211 (+ 144us) client_negotiation.cc:769] Sending connection context
0807 15:11:52.385315 (+ 104us) client_negotiation.cc:240] Negotiation successful
0807 15:11:52.385556 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":109,"threads_started":1}
I20260807 15:11:52.386425 8031 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.194.61" port: 40955 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 15:11:52 dist-test-slave-42z9 krb5kdc[6944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115511, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.29.194.60@KRBTEST.COM
Aug 07 15:11:52 dist-test-slave-42z9 krb5kdc[6944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115512, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.194.60@KRBTEST.COM for oryx/127.29.194.61@KRBTEST.COM
I20260807 15:11:52.391319 8036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:52.381417 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33740 (local address 127.29.194.60:44209)
0807 15:11:52.381670 (+ 253us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:52.381677 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:52.381700 (+ 23us) server_negotiation.cc:407] Connection header received
0807 15:11:52.381721 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:52.381723 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:52.381737 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:52.381818 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:11:52.382567 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:52.383487 (+ 920us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:52.384469 (+ 982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:52.384926 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:52.385021 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:52.387848 (+ 2827us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:11:52.387858 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:11:52.387858 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:11:52.387867 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:11:52.389720 (+ 1853us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:52.390161 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:52.390165 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:52.390166 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:52.390231 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:52.390631 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:52.390632 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:52.390632 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:52.390803 (+ 171us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:11:52.390925 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0807 15:11:52.391085 (+ 160us) server_negotiation.cc:299] Negotiation successful
0807 15:11:52.391193 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":215,"thread_start_us":162,"threads_started":1}
I20260807 15:11:52.394075 7929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:52.386841 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51680 (local address 127.29.194.61:40955)
0807 15:11:52.386935 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:52.386938 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:52.386949 (+ 11us) server_negotiation.cc:407] Connection header received
0807 15:11:52.386984 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:52.386985 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:52.387002 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:52.387074 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:11:52.387508 (+ 434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:52.388159 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:52.389113 (+ 954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:52.389316 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:52.389378 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:52.391566 (+ 2188us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:11:52.391568 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:11:52.391569 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:11:52.391578 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:11:52.392911 (+ 1333us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:52.393297 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:52.393298 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:52.393299 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:52.393347 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:52.393659 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:52.393659 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:52.393659 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:52.393770 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:11:52.393830 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0807 15:11:52.393932 (+ 102us) server_negotiation.cc:299] Negotiation successful
0807 15:11:52.394009 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260807 15:11:52.394076 8034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:52.386736 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.61:40955 (local address 127.0.0.1:51680)
0807 15:11:52.386808 (+ 72us) negotiation.cc:107] Waiting for socket to connect
0807 15:11:52.386822 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 15:11:52.386870 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:11:52.387127 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:11:52.387129 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:11:52.387262 (+ 133us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:11:52.387332 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:52.387334 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:52.388250 (+ 916us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:52.388251 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:52.389010 (+ 759us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:52.389014 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:52.389399 (+ 385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:52.389400 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:52.389498 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:52.389851 (+ 353us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:11:52.389856 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:11:52.391454 (+ 1598us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:11:52.393047 (+ 1593us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:11:52.393048 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:11:52.393049 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:11:52.393230 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:11:52.393522 (+ 292us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:11:52.393523 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:11:52.393524 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:11:52.393594 (+ 70us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:11:52.393828 (+ 234us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:11:52.393829 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:11:52.393867 (+ 38us) client_negotiation.cc:769] Sending connection context
0807 15:11:52.393914 (+ 47us) client_negotiation.cc:240] Negotiation successful
0807 15:11:52.394004 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43}
I20260807 15:11:52.394596 8031 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.194.60" port: 44209 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 15:11:52 dist-test-slave-42z9 krb5kdc[6944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115512, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.194.60@KRBTEST.COM for oryx/127.29.194.60@KRBTEST.COM
I20260807 15:11:52.401487 8036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:52.394906 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33742 (local address 127.29.194.60:44209)
0807 15:11:52.394949 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:52.394951 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:52.394958 (+ 7us) server_negotiation.cc:407] Connection header received
0807 15:11:52.395005 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:52.395005 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:52.395015 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:52.395097 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:11:52.395329 (+ 232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:52.395810 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:52.396410 (+ 600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:52.396621 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:52.396678 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:52.398730 (+ 2052us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:11:52.398732 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:11:52.398732 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:11:52.398739 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:11:52.400309 (+ 1570us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:52.400671 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:52.400672 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:52.400672 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:52.400725 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:52.400903 (+ 178us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:52.400904 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:52.400904 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:52.401041 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:11:52.401103 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0807 15:11:52.401294 (+ 191us) server_negotiation.cc:299] Negotiation successful
0807 15:11:52.401376 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260807 15:11:52.401612 8034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:52.394858 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.60:44209 (local address 127.0.0.1:33742)
0807 15:11:52.394902 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0807 15:11:52.394913 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 15:11:52.394953 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:11:52.395100 (+ 147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:11:52.395101 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:11:52.395235 (+ 134us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:11:52.395267 (+ 32us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:52.395269 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:52.395879 (+ 610us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:52.395880 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:52.396343 (+ 463us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:52.396345 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:52.396680 (+ 335us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:52.396680 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:52.396725 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:52.397137 (+ 412us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:11:52.397142 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:11:52.398610 (+ 1468us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:11:52.400418 (+ 1808us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:11:52.400419 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:11:52.400419 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:11:52.400590 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:11:52.400778 (+ 188us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:11:52.400778 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:11:52.400779 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:11:52.400829 (+ 50us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:11:52.401137 (+ 308us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:11:52.401138 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:11:52.401188 (+ 50us) client_negotiation.cc:769] Sending connection context
0807 15:11:52.401269 (+ 81us) client_negotiation.cc:240] Negotiation successful
0807 15:11:52.401499 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":25,"mutex_wait_us":83}
I20260807 15:11:52.404031 8031 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e5603c6f9ea84f1a9510499da5950324: Bootstrap starting.
I20260807 15:11:52.404939 8031 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e5603c6f9ea84f1a9510499da5950324: Neither blocks nor log segments found. Creating new log.
I20260807 15:11:52.405185 8031 log.cc:824] T 00000000000000000000000000000000 P e5603c6f9ea84f1a9510499da5950324: Log is configured to *not* fsync() on all Append() calls
I20260807 15:11:52.405972 8031 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e5603c6f9ea84f1a9510499da5950324: No bootstrap required, opened a new log
I20260807 15:11:52.407583 8031 raft_consensus.cc:348] T 00000000000000000000000000000000 P e5603c6f9ea84f1a9510499da5950324 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ebd9481251f4a29ba10f88819ba9ae2" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 39853 } } peers { permanent_uuid: "8e78c2a81ab3476b8cbeff1833835aeb" member_type: VOTER last_known_addr { host: "127.29.194.61" port: 40955 } } peers { permanent_uuid: "e5603c6f9ea84f1a9510499da5950324" member_type: VOTER last_known_addr { host: "127.29.194.60" port: 44209 } }
I20260807 15:11:52.407725 8031 raft_consensus.cc:374] T 00000000000000000000000000000000 P e5603c6f9ea84f1a9510499da5950324 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:11:52.407748 8031 raft_consensus.cc:729] T 00000000000000000000000000000000 P e5603c6f9ea84f1a9510499da5950324 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5603c6f9ea84f1a9510499da5950324, State: Initialized, Role: FOLLOWER
I20260807 15:11:52.407866 8031 consensus_queue.cc:260] T 00000000000000000000000000000000 P e5603c6f9ea84f1a9510499da5950324 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ebd9481251f4a29ba10f88819ba9ae2" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 39853 } } peers { permanent_uuid: "8e78c2a81ab3476b8cbeff1833835aeb" member_type: VOTER last_known_addr { host: "127.29.194.61" port: 40955 } } peers { permanent_uuid: "e5603c6f9ea84f1a9510499da5950324" member_type: VOTER last_known_addr { host: "127.29.194.60" port: 44209 } }
I20260807 15:11:52.408218 8031 sys_catalog.cc:564] T 00000000000000000000000000000000 P e5603c6f9ea84f1a9510499da5950324 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:11:52.408186 8038 sys_catalog.cc:455] T 00000000000000000000000000000000 P e5603c6f9ea84f1a9510499da5950324 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ebd9481251f4a29ba10f88819ba9ae2" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 39853 } } peers { permanent_uuid: "8e78c2a81ab3476b8cbeff1833835aeb" member_type: VOTER last_known_addr { host: "127.29.194.61" port: 40955 } } peers { permanent_uuid: "e5603c6f9ea84f1a9510499da5950324" member_type: VOTER last_known_addr { host: "127.29.194.60" port: 44209 } } }
I20260807 15:11:52.408268 8038 sys_catalog.cc:458] T 00000000000000000000000000000000 P e5603c6f9ea84f1a9510499da5950324 [sys.catalog]: This master's current role is: FOLLOWER
W20260807 15:11:52.410297 8054 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e5603c6f9ea84f1a9510499da5950324: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 15:11:52.410401 8054 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 15:11:52.412923 7936 master_runner.cc:428] Detected that this master e5603c6f9ea84f1a9510499da5950324 is joining an existing cluster
I20260807 15:11:52.413007 7936 master_runner.cc:432] Initiating AddMaster RPC to add 127.29.194.60:44209
I20260807 15:11:52.421193 7061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:52.414293 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43356 (local address 127.29.194.62:39853)
0807 15:11:52.414338 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:52.414340 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:52.414777 (+ 437us) server_negotiation.cc:407] Connection header received
0807 15:11:52.414870 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:52.414871 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:52.414882 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:52.414910 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:11:52.415390 (+ 480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:52.416129 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:52.417309 (+ 1180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:52.417643 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:52.417682 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:52.418851 (+ 1169us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:11:52.418852 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:11:52.418853 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:11:52.418861 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:11:52.420141 (+ 1280us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:52.420466 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:52.420467 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:52.420468 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:52.420519 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:52.420697 (+ 178us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:52.420697 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:52.420698 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:52.420811 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:11:52.420868 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0807 15:11:52.421010 (+ 142us) server_negotiation.cc:299] Negotiation successful
0807 15:11:52.421093 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260807 15:11:52.421211 8063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:52.414429 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:39853 (local address 127.0.0.1:43356)
0807 15:11:52.414726 (+ 297us) negotiation.cc:107] Waiting for socket to connect
0807 15:11:52.414734 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0807 15:11:52.414799 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:11:52.414971 (+ 172us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:11:52.414972 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:11:52.415220 (+ 248us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:11:52.415292 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:52.415295 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:52.416205 (+ 910us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:52.416206 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:52.417207 (+ 1001us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:52.417210 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:52.417771 (+ 561us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:52.417771 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:52.417853 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:52.418323 (+ 470us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:11:52.418328 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:11:52.418721 (+ 393us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:11:52.420223 (+ 1502us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:11:52.420225 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:11:52.420225 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:11:52.420401 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:11:52.420575 (+ 174us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:11:52.420576 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:11:52.420576 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:11:52.420646 (+ 70us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:11:52.420867 (+ 221us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:11:52.420868 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:11:52.420922 (+ 54us) client_negotiation.cc:769] Sending connection context
0807 15:11:52.420981 (+ 59us) client_negotiation.cc:240] Negotiation successful
0807 15:11:52.421095 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":285,"mutex_wait_us":79,"thread_start_us":86,"threads_started":1}
Aug 07 15:11:52 dist-test-slave-42z9 krb5kdc[6944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115511, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.194.62@KRBTEST.COM for oryx/127.29.194.60@KRBTEST.COM
I20260807 15:11:52.441155 7801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:52.424388 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.60:44209 (local address 127.0.0.1:33768)
0807 15:11:52.424478 (+ 90us) negotiation.cc:107] Waiting for socket to connect
0807 15:11:52.424493 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 15:11:52.424557 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:11:52.425031 (+ 474us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:11:52.425032 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:11:52.425243 (+ 211us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:11:52.425326 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:52.425328 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:52.426681 (+ 1353us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:52.426682 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:52.427223 (+ 541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:52.427224 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:52.427760 (+ 536us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:52.427761 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:52.427845 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:52.428304 (+ 459us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:11:52.428310 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:11:52.429852 (+ 1542us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:11:52.439938 (+ 10086us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:11:52.439941 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:11:52.439941 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:11:52.440195 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:11:52.440450 (+ 255us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:11:52.440451 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:11:52.440451 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:11:52.440521 (+ 70us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:11:52.440839 (+ 318us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:11:52.440840 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:11:52.440893 (+ 53us) client_negotiation.cc:769] Sending connection context
0807 15:11:52.440942 (+ 49us) client_negotiation.cc:240] Negotiation successful
0807 15:11:52.441069 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":57}
I20260807 15:11:52.441632 8065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:52.424487 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33768 (local address 127.29.194.60:44209)
0807 15:11:52.424841 (+ 354us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:52.424844 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:52.424853 (+ 9us) server_negotiation.cc:407] Connection header received
0807 15:11:52.424897 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:52.424898 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:52.424921 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:52.425008 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:11:52.425415 (+ 407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:52.426567 (+ 1152us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:52.427304 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:52.427647 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:52.427717 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:52.429968 (+ 2251us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:11:52.429970 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:11:52.429971 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:11:52.429979 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:11:52.439803 (+ 9824us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:52.440285 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:52.440286 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:52.440287 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:52.440357 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:52.440595 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:52.440596 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:52.440596 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:52.440759 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:11:52.441395 (+ 636us) server_negotiation.cc:1035] Waiting for connection context
0807 15:11:52.441461 (+ 66us) server_negotiation.cc:299] Negotiation successful
0807 15:11:52.441554 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":305,"thread_start_us":255,"threads_started":1}
I20260807 15:11:52.442198 7012 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.29.194.60:44209: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "e5603c6f9ea84f1a9510499da5950324"
member_type: NON_VOTER
last_known_addr {
host: "127.29.194.60"
port: 44209
}
attrs {
promote: true
}
}
dest_uuid: "4ebd9481251f4a29ba10f88819ba9ae2"
cas_config_opid_index: 5
I20260807 15:11:52.442469 7929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:52.414483 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51696 (local address 127.29.194.61:40955)
0807 15:11:52.414524 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:52.414526 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:52.414677 (+ 151us) server_negotiation.cc:407] Connection header received
0807 15:11:52.414707 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:52.414708 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:52.414721 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:52.414781 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:11:52.415409 (+ 628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:52.416152 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:52.417356 (+ 1204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:52.417701 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:52.417747 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:52.439350 (+ 21603us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:11:52.439353 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:11:52.439353 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:11:52.439364 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:11:52.440841 (+ 1477us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:52.441403 (+ 562us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:52.441404 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:52.441404 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:52.441507 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:52.441888 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:52.441889 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:52.441889 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:52.442028 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:11:52.442087 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0807 15:11:52.442291 (+ 204us) server_negotiation.cc:299] Negotiation successful
0807 15:11:52.442389 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260807 15:11:52.442474 8062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:52.414248 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.61:40955 (local address 127.0.0.1:51696)
0807 15:11:52.414576 (+ 328us) negotiation.cc:107] Waiting for socket to connect
0807 15:11:52.414587 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 15:11:52.414654 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:11:52.414845 (+ 191us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:11:52.414846 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:11:52.415184 (+ 338us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:11:52.415299 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:52.415302 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:52.416251 (+ 949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:52.416252 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:52.417252 (+ 1000us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:52.417256 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:52.417874 (+ 618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:52.417875 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:52.417943 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:52.438135 (+ 20192us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:11:52.438146 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:11:52.439184 (+ 1038us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:11:52.440961 (+ 1777us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:11:52.440962 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:11:52.440963 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:11:52.441211 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:11:52.441726 (+ 515us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:11:52.441736 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:11:52.441736 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:11:52.441812 (+ 76us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:11:52.442112 (+ 300us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:11:52.442113 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:11:52.442168 (+ 55us) client_negotiation.cc:769] Sending connection context
0807 15:11:52.442234 (+ 66us) client_negotiation.cc:240] Negotiation successful
0807 15:11:52.442351 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":302,"mutex_wait_us":47,"thread_start_us":98,"threads_started":1}
I20260807 15:11:52.442467 7012 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4ebd9481251f4a29ba10f88819ba9ae2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "4ebd9481251f4a29ba10f88819ba9ae2" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 39853 } } peers { permanent_uuid: "8e78c2a81ab3476b8cbeff1833835aeb" member_type: NON_VOTER last_known_addr { host: "127.29.194.61" port: 40955 } attrs { promote: true } } peers { permanent_uuid: "e5603c6f9ea84f1a9510499da5950324" member_type: NON_VOTER last_known_addr { host: "127.29.194.60" port: 44209 } attrs { promote: true } }
I20260807 15:11:52.443053 30472 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7fZSIL/build/release/bin/kudu with pid 7092
I20260807 15:11:52.443086 7803 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 4ebd9481251f4a29ba10f88819ba9ae2 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER e5603c6f9ea84f1a9510499da5950324 (127.29.194.60) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "4ebd9481251f4a29ba10f88819ba9ae2" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 39853 } } peers { permanent_uuid: "8e78c2a81ab3476b8cbeff1833835aeb" member_type: NON_VOTER last_known_addr { host: "127.29.194.61" port: 40955 } attrs { promote: true } } peers { permanent_uuid: "e5603c6f9ea84f1a9510499da5950324" member_type: NON_VOTER last_known_addr { host: "127.29.194.60" port: 44209 } attrs { promote: true } } }
I20260807 15:11:52.444108 7803 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.29.194.60:44209
I20260807 15:11:52.444195 7803 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4ebd9481251f4a29ba10f88819ba9ae2 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "4ebd9481251f4a29ba10f88819ba9ae2" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "4ebd9481251f4a29ba10f88819ba9ae2" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 39853 } } peers { permanent_uuid: "8e78c2a81ab3476b8cbeff1833835aeb" member_type: NON_VOTER last_known_addr { host: "127.29.194.61" port: 40955 } attrs { promote: true } } peers { permanent_uuid: "e5603c6f9ea84f1a9510499da5950324" member_type: NON_VOTER last_known_addr { host: "127.29.194.60" port: 44209 } attrs { promote: true } } }
I20260807 15:11:52.444267 7803 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4ebd9481251f4a29ba10f88819ba9ae2 [sys.catalog]: This master's current role is: LEADER
I20260807 15:11:52.446224 7995 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e5603c6f9ea84f1a9510499da5950324 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:11:52.446282 7936 master.cc:537] Master@127.29.194.60:44209 shutting down...
I20260807 15:11:52.447357 7995 raft_consensus.cc:1261] T 00000000000000000000000000000000 P e5603c6f9ea84f1a9510499da5950324 [term 1 FOLLOWER]: Refusing update from remote peer 4ebd9481251f4a29ba10f88819ba9ae2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260807 15:11:52.449008 8064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:52.414366 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.60:44209 (local address 127.0.0.1:33752)
0807 15:11:52.414576 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0807 15:11:52.414587 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 15:11:52.414654 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:11:52.414833 (+ 179us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:11:52.414834 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:11:52.415178 (+ 344us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:11:52.415304 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:52.415305 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:52.416199 (+ 894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:52.416200 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:52.417238 (+ 1038us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:52.417240 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:52.417769 (+ 529us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:52.417770 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:52.417870 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:52.438363 (+ 20493us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:11:52.438371 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:11:52.439159 (+ 788us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:11:52.442110 (+ 2951us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:11:52.442110 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:11:52.442111 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:11:52.447558 (+ 5447us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:11:52.448103 (+ 545us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:11:52.448104 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:11:52.448104 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:11:52.448173 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:11:52.448620 (+ 447us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:11:52.448621 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:11:52.448665 (+ 44us) client_negotiation.cc:769] Sending connection context
0807 15:11:52.448729 (+ 64us) client_negotiation.cc:240] Negotiation successful
0807 15:11:52.448866 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":190,"mutex_wait_us":140,"thread_start_us":76,"threads_started":1}
I20260807 15:11:52.449012 8036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:52.414273 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33752 (local address 127.29.194.60:44209)
0807 15:11:52.414333 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:52.414335 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:52.414660 (+ 325us) server_negotiation.cc:407] Connection header received
0807 15:11:52.414727 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:52.414727 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:52.414736 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:52.414763 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:11:52.415372 (+ 609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:52.416113 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:52.417309 (+ 1196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:52.417640 (+ 331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:52.417682 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:52.439289 (+ 21607us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:11:52.439291 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:11:52.439291 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:11:52.439298 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:11:52.441649 (+ 2351us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:52.447652 (+ 6003us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:52.447653 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:52.447654 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:52.447720 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:52.448235 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:52.448235 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:52.448236 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:52.448368 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:11:52.448408 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0807 15:11:52.448765 (+ 357us) server_negotiation.cc:299] Negotiation successful
0807 15:11:52.448924 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":89,"server-negotiator.queue_time_us":37}
I20260807 15:11:52.449723 7802 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 4ebd9481251f4a29ba10f88819ba9ae2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5603c6f9ea84f1a9510499da5950324" member_type: NON_VOTER last_known_addr { host: "127.29.194.60" port: 44209 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
W20260807 15:11:52.450430 6986 consensus_peers.cc:487] T 00000000000000000000000000000000 P 4ebd9481251f4a29ba10f88819ba9ae2 -> Peer e5603c6f9ea84f1a9510499da5950324 (127.29.194.60:44209): Couldn't send request to peer e5603c6f9ea84f1a9510499da5950324. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260807 15:11:52.452963 7801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:52.445771 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.61:40955 (local address 127.0.0.1:51698)
0807 15:11:52.446724 (+ 953us) negotiation.cc:107] Waiting for socket to connect
0807 15:11:52.446737 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 15:11:52.446788 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:11:52.450287 (+ 3499us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:11:52.450289 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:11:52.450291 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:11:52.450342 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:52.450344 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:52.451069 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:52.451070 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:52.451923 (+ 853us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:52.451926 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:52.452708 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:52.452709 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:52.452765 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:52.452791 (+ 26us) client_negotiation.cc:769] Sending connection context
0807 15:11:52.452840 (+ 49us) client_negotiation.cc:240] Negotiation successful
0807 15:11:52.452887 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":931}
I20260807 15:11:52.453135 7936 raft_consensus.cc:2229] T 00000000000000000000000000000000 P e5603c6f9ea84f1a9510499da5950324 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 15:11:52.453182 7929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:52.449566 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51698 (local address 127.29.194.61:40955)
0807 15:11:52.449885 (+ 319us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:52.449887 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:52.449897 (+ 10us) server_negotiation.cc:407] Connection header received
0807 15:11:52.449922 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:52.449923 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:52.449948 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:52.450010 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:11:52.450427 (+ 417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:52.450995 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:52.452011 (+ 1016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:52.452529 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:52.452619 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:52.452678 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0807 15:11:52.453067 (+ 389us) server_negotiation.cc:299] Negotiation successful
0807 15:11:52.453115 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":293}
I20260807 15:11:52.453241 7936 raft_consensus.cc:2258] T 00000000000000000000000000000000 P e5603c6f9ea84f1a9510499da5950324 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:11:52.453289 7936 tablet_replica.cc:330] T 00000000000000000000000000000000 P e5603c6f9ea84f1a9510499da5950324: stopping tablet replica
I20260807 15:11:52.453408 7868 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 8e78c2a81ab3476b8cbeff1833835aeb [term 1 LEARNER]: Refusing update from remote peer 4ebd9481251f4a29ba10f88819ba9ae2: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260807 15:11:52.453629 8069 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 4ebd9481251f4a29ba10f88819ba9ae2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e78c2a81ab3476b8cbeff1833835aeb" member_type: NON_VOTER last_known_addr { host: "127.29.194.61" port: 40955 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260807 15:11:52.454252 7868 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 8e78c2a81ab3476b8cbeff1833835aeb [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER e5603c6f9ea84f1a9510499da5950324 (127.29.194.60) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "4ebd9481251f4a29ba10f88819ba9ae2" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 39853 } } peers { permanent_uuid: "8e78c2a81ab3476b8cbeff1833835aeb" member_type: NON_VOTER last_known_addr { host: "127.29.194.61" port: 40955 } attrs { promote: true } } peers { permanent_uuid: "e5603c6f9ea84f1a9510499da5950324" member_type: NON_VOTER last_known_addr { host: "127.29.194.60" port: 44209 } attrs { promote: true } } }
I20260807 15:11:52.454571 30472 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7fZSIL/build/release/bin/kudu
/tmp/dist-test-task7fZSIL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115480851507-30472-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115480851507-30472-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115480851507-30472-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115480851507-30472-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.194.1:34085
--local_ip_for_outbound_sockets=127.29.194.1
--tserver_master_addrs=127.29.194.62:39853
--webserver_port=40935
--webserver_interface=127.29.194.1
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115480851507-30472-0/krb5kdc/oryx_127.29.194.1.keytab
--principal=oryx/127.29.194.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.194.20:45569
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.29.194.62:39853,127.29.194.61:40955
--tserver_master_addrs=127.29.194.62:39853,127.29.194.61:40955,127.29.194.60:44209 with env {KRB5CCNAME=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115480851507-30472-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115480851507-30472-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115480851507-30472-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 15:11:52.455464 8071 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8e78c2a81ab3476b8cbeff1833835aeb [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "4ebd9481251f4a29ba10f88819ba9ae2" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "4ebd9481251f4a29ba10f88819ba9ae2" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 39853 } } peers { permanent_uuid: "8e78c2a81ab3476b8cbeff1833835aeb" member_type: NON_VOTER last_known_addr { host: "127.29.194.61" port: 40955 } attrs { promote: true } } peers { permanent_uuid: "e5603c6f9ea84f1a9510499da5950324" member_type: NON_VOTER last_known_addr { host: "127.29.194.60" port: 44209 } attrs { promote: true } } }
I20260807 15:11:52.455626 8071 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8e78c2a81ab3476b8cbeff1833835aeb [sys.catalog]: This master's current role is: LEARNER
I20260807 15:11:52.455677 7914 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8e78c2a81ab3476b8cbeff1833835aeb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4ebd9481251f4a29ba10f88819ba9ae2. Latest consensus state: current_term: 1 leader_uuid: "4ebd9481251f4a29ba10f88819ba9ae2" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "4ebd9481251f4a29ba10f88819ba9ae2" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 39853 } } peers { permanent_uuid: "8e78c2a81ab3476b8cbeff1833835aeb" member_type: NON_VOTER last_known_addr { host: "127.29.194.61" port: 40955 } attrs { promote: true } } peers { permanent_uuid: "e5603c6f9ea84f1a9510499da5950324" member_type: NON_VOTER last_known_addr { host: "127.29.194.60" port: 44209 } attrs { promote: true } } }
I20260807 15:11:52.455744 7914 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8e78c2a81ab3476b8cbeff1833835aeb [sys.catalog]: This master's current role is: LEARNER
W20260807 15:11:52.456310 7961 connection.cc:441] server connection from 127.0.0.1:33752 torn down before Call kudu.master.MasterService.ConnectToMaster from 127.0.0.1:33752 (request call id 2) could send its response
I20260807 15:11:52.457146 7802 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 4ebd9481251f4a29ba10f88819ba9ae2: attempting to promote NON_VOTER 8e78c2a81ab3476b8cbeff1833835aeb to VOTER
I20260807 15:11:52.457381 7802 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4ebd9481251f4a29ba10f88819ba9ae2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "4ebd9481251f4a29ba10f88819ba9ae2" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 39853 } } peers { permanent_uuid: "8e78c2a81ab3476b8cbeff1833835aeb" member_type: VOTER last_known_addr { host: "127.29.194.61" port: 40955 } attrs { promote: false } } peers { permanent_uuid: "e5603c6f9ea84f1a9510499da5950324" member_type: NON_VOTER last_known_addr { host: "127.29.194.60" port: 44209 } attrs { promote: true } }
I20260807 15:11:52.458950 7868 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 8e78c2a81ab3476b8cbeff1833835aeb [term 1 LEARNER]: Refusing update from remote peer 4ebd9481251f4a29ba10f88819ba9ae2: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260807 15:11:52.459013 7801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:52.458832 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.60:44209 (local address 127.0.0.1:33778)
0807 15:11:52.458885 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0807 15:11:52.458938 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.194.60:44209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260807 15:11:52.459678 7801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:52.459553 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.60:44209 (local address 127.0.0.1:33794)
0807 15:11:52.459601 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0807 15:11:52.459628 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.194.60:44209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260807 15:11:52.459619 7803 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 4ebd9481251f4a29ba10f88819ba9ae2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e78c2a81ab3476b8cbeff1833835aeb" member_type: VOTER last_known_addr { host: "127.29.194.61" port: 40955 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
W20260807 15:11:52.459759 6986 consensus_peers.cc:487] T 00000000000000000000000000000000 P 4ebd9481251f4a29ba10f88819ba9ae2 -> Peer e5603c6f9ea84f1a9510499da5950324 (127.29.194.60:44209): Couldn't send request to peer e5603c6f9ea84f1a9510499da5950324. Status: Network error: Client connection negotiation failed: client connection to 127.29.194.60:44209: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260807 15:11:52.460595 7803 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 4ebd9481251f4a29ba10f88819ba9ae2 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 8e78c2a81ab3476b8cbeff1833835aeb (127.29.194.61) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "4ebd9481251f4a29ba10f88819ba9ae2" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 39853 } } peers { permanent_uuid: "8e78c2a81ab3476b8cbeff1833835aeb" member_type: VOTER last_known_addr { host: "127.29.194.61" port: 40955 } attrs { promote: false } } peers { permanent_uuid: "e5603c6f9ea84f1a9510499da5950324" member_type: NON_VOTER last_known_addr { host: "127.29.194.60" port: 44209 } attrs { promote: true } } }
I20260807 15:11:52.462069 7803 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4ebd9481251f4a29ba10f88819ba9ae2 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "4ebd9481251f4a29ba10f88819ba9ae2" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "4ebd9481251f4a29ba10f88819ba9ae2" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 39853 } } peers { permanent_uuid: "8e78c2a81ab3476b8cbeff1833835aeb" member_type: VOTER last_known_addr { host: "127.29.194.61" port: 40955 } attrs { promote: false } } peers { permanent_uuid: "e5603c6f9ea84f1a9510499da5950324" member_type: NON_VOTER last_known_addr { host: "127.29.194.60" port: 44209 } attrs { promote: true } } }
I20260807 15:11:52.462177 7803 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4ebd9481251f4a29ba10f88819ba9ae2 [sys.catalog]: This master's current role is: LEADER
I20260807 15:11:52.462234 7803 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4ebd9481251f4a29ba10f88819ba9ae2 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "4ebd9481251f4a29ba10f88819ba9ae2" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "4ebd9481251f4a29ba10f88819ba9ae2" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 39853 } } peers { permanent_uuid: "8e78c2a81ab3476b8cbeff1833835aeb" member_type: VOTER last_known_addr { host: "127.29.194.61" port: 40955 } attrs { promote: false } } peers { permanent_uuid: "e5603c6f9ea84f1a9510499da5950324" member_type: NON_VOTER last_known_addr { host: "127.29.194.60" port: 44209 } attrs { promote: true } } }
I20260807 15:11:52.462277 7803 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4ebd9481251f4a29ba10f88819ba9ae2 [sys.catalog]: This master's current role is: LEADER
I20260807 15:11:52.462621 7868 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 8e78c2a81ab3476b8cbeff1833835aeb [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 8e78c2a81ab3476b8cbeff1833835aeb (127.29.194.61) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "4ebd9481251f4a29ba10f88819ba9ae2" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 39853 } } peers { permanent_uuid: "8e78c2a81ab3476b8cbeff1833835aeb" member_type: VOTER last_known_addr { host: "127.29.194.61" port: 40955 } attrs { promote: false } } peers { permanent_uuid: "e5603c6f9ea84f1a9510499da5950324" member_type: NON_VOTER last_known_addr { host: "127.29.194.60" port: 44209 } attrs { promote: true } } }
I20260807 15:11:52.463536 7914 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8e78c2a81ab3476b8cbeff1833835aeb [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "4ebd9481251f4a29ba10f88819ba9ae2" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "4ebd9481251f4a29ba10f88819ba9ae2" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 39853 } } peers { permanent_uuid: "8e78c2a81ab3476b8cbeff1833835aeb" member_type: VOTER last_known_addr { host: "127.29.194.61" port: 40955 } attrs { promote: false } } peers { permanent_uuid: "e5603c6f9ea84f1a9510499da5950324" member_type: NON_VOTER last_known_addr { host: "127.29.194.60" port: 44209 } attrs { promote: true } } }
I20260807 15:11:52.463619 7914 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8e78c2a81ab3476b8cbeff1833835aeb [sys.catalog]: This master's current role is: FOLLOWER
I20260807 15:11:52.468374 7936 master.cc:552] Master@127.29.194.60:44209 shutdown complete.
I20260807 15:11:52.468432 7936 master_runner.cc:305] Clearing existing system tablet
I20260807 15:11:52.468564 7936 fs_manager.cc:359] Metadata directory not provided
I20260807 15:11:52.468616 7936 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:11:52.469220 7936 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.001s sys 0.000s
I20260807 15:11:52.469712 8076 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:11:52.469805 7936 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115480851507-30472-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:11:52.469851 7936 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260807 15:11:52.469875 7936 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115480851507-30472-0/minicluster-data/master-2/data,/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115480851507-30472-0/minicluster-data/master-2/wal
uuid: "e5603c6f9ea84f1a9510499da5950324"
format_stamp: "Formatted at 2026-08-07 15:11:52 on dist-test-slave-42z9"
server_key: "eb40f07c6ebf8c88cb03c225267bfdc8"
server_key_iv: "cbb7178be5d1a530b9f26031cd9d71ea"
server_key_version: "encryptionkey@0"
I20260807 15:11:52.470532 7936 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P e5603c6f9ea84f1a9510499da5950324: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:11:52.471912 7936 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P e5603c6f9ea84f1a9510499da5950324: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260807 15:11:52.471979 7936 log.cc:1196] T 00000000000000000000000000000000 P e5603c6f9ea84f1a9510499da5950324: Deleting WAL directory at /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115480851507-30472-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260807 15:11:52.472194 7936 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P e5603c6f9ea84f1a9510499da5950324: Deleting consensus metadata
I20260807 15:11:52.472324 7936 master_runner.cc:315] Copying system tablet from 127.29.194.62:39853
I20260807 15:11:52.472939 7936 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P e5603c6f9ea84f1a9510499da5950324: tablet copy: Beginning tablet copy session from remote peer at address 127.29.194.62:39853
I20260807 15:11:52.479372 7061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:52.473406 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43362 (local address 127.29.194.62:39853)
0807 15:11:52.473476 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:52.473478 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:52.473487 (+ 9us) server_negotiation.cc:407] Connection header received
0807 15:11:52.473645 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:52.473647 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:52.473658 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:52.473704 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:11:52.474016 (+ 312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:52.474521 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:52.475219 (+ 698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:52.475423 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:52.475478 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:52.476392 (+ 914us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:11:52.476393 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:11:52.476394 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:11:52.476400 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:11:52.477760 (+ 1360us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:52.478065 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:52.478066 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:52.478066 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:52.478119 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:52.478420 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:52.478421 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:52.478422 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:52.478688 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:11:52.478905 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0807 15:11:52.479061 (+ 156us) server_negotiation.cc:299] Negotiation successful
0807 15:11:52.479150 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260807 15:11:52.479427 8085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:52.473260 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:39853 (local address 127.0.0.1:43362)
0807 15:11:52.473380 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0807 15:11:52.473391 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 15:11:52.473579 (+ 188us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:11:52.473708 (+ 129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:11:52.473709 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:11:52.473893 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:11:52.473948 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:52.473950 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:52.474597 (+ 647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:52.474598 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:52.475147 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:52.475149 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:52.475485 (+ 336us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:52.475486 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:52.475549 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:52.475990 (+ 441us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:11:52.475995 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:11:52.476307 (+ 312us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:11:52.477831 (+ 1524us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:11:52.477832 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:11:52.477832 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:11:52.477999 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:11:52.478172 (+ 173us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:11:52.478172 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:11:52.478173 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:11:52.478232 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:11:52.478899 (+ 667us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:11:52.478900 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:11:52.478959 (+ 59us) client_negotiation.cc:769] Sending connection context
0807 15:11:52.479024 (+ 65us) client_negotiation.cc:240] Negotiation successful
0807 15:11:52.479151 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":95,"thread_start_us":77,"threads_started":1}
I20260807 15:11:52.479977 7035 tablet_copy_service.cc:140] P 4ebd9481251f4a29ba10f88819ba9ae2: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer e5603c6f9ea84f1a9510499da5950324 ({username='oryx', principal='oryx/127.29.194.60@KRBTEST.COM'} at 127.0.0.1:43362)
I20260807 15:11:52.480034 7035 tablet_copy_service.cc:161] P 4ebd9481251f4a29ba10f88819ba9ae2: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer e5603c6f9ea84f1a9510499da5950324 at {username='oryx', principal='oryx/127.29.194.60@KRBTEST.COM'} at 127.0.0.1:43362: session id = e5603c6f9ea84f1a9510499da5950324-00000000000000000000000000000000
I20260807 15:11:52.480528 7035 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 4ebd9481251f4a29ba10f88819ba9ae2: Tablet Copy: opened 0 blocks and 1 log segments
I20260807 15:11:52.481030 7936 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260807 15:11:52.482085 7936 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P e5603c6f9ea84f1a9510499da5950324: tablet copy: Starting download of 0 data blocks...
I20260807 15:11:52.482236 7936 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P e5603c6f9ea84f1a9510499da5950324: tablet copy: Starting download of 1 WAL segments...
I20260807 15:11:52.482941 7936 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P e5603c6f9ea84f1a9510499da5950324: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260807 15:11:52.483779 7035 tablet_copy_service.cc:342] P 4ebd9481251f4a29ba10f88819ba9ae2: Request end of tablet copy session e5603c6f9ea84f1a9510499da5950324-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.29.194.60@KRBTEST.COM'} at 127.0.0.1:43362
I20260807 15:11:52.483839 7035 tablet_copy_service.cc:433] P 4ebd9481251f4a29ba10f88819ba9ae2: ending tablet copy session e5603c6f9ea84f1a9510499da5950324-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer e5603c6f9ea84f1a9510499da5950324
I20260807 15:11:52.486752 7936 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:11:52.488123 8087 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:11:52.488270 7936 server_base.cc:1034] running on GCE node
W20260807 15:11:52.488155 8090 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:11:52.489688 8088 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:11:52.489888 7936 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:11:52.490062 7936 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:11:52.491205 7936 hybrid_clock.cc:648] HybridClock initialized: now 1786115512491198 us; error 29 us; skew 500 ppm
Aug 07 15:11:52 dist-test-slave-42z9 krb5kdc[6944](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115512, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.194.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:11:52.492556 7936 init.cc:377] Logged in from keytab as oryx/127.29.194.60@KRBTEST.COM (short username oryx)
I20260807 15:11:52.492833 7936 webserver.cc:460] Webserver started at http://127.29.194.60:40295/ using document root <none> and password file <none>
I20260807 15:11:52.492949 7936 fs_manager.cc:359] Metadata directory not provided
I20260807 15:11:52.492982 7936 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:11:52.493607 7936 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.000s sys 0.000s
I20260807 15:11:52.494109 8100 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:11:52.494206 7936 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260807 15:11:52.494242 7936 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115480851507-30472-0/minicluster-data/master-2/data,/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115480851507-30472-0/minicluster-data/master-2/wal
uuid: "e5603c6f9ea84f1a9510499da5950324"
format_stamp: "Formatted at 2026-08-07 15:11:52 on dist-test-slave-42z9"
server_key: "eb40f07c6ebf8c88cb03c225267bfdc8"
server_key_iv: "cbb7178be5d1a530b9f26031cd9d71ea"
server_key_version: "encryptionkey@0"
I20260807 15:11:52.494302 7936 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115480851507-30472-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115480851507-30472-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115480851507-30472-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:11:52.504204 7936 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:11:52.506258 7936 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:11:52.509294 7936 rpc_server.cc:307] RPC server started. Bound to: 127.29.194.60:44209
I20260807 15:11:52.509769 7936 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115480851507-30472-0/minicluster-data/master-2/data/info.pb
I20260807 15:11:52.510013 8172 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.194.60:44209 every 8 connection(s)
I20260807 15:11:52.514130 8173 sys_catalog.cc:263] Verifying existing consensus state
I20260807 15:11:52.514911 8173 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e5603c6f9ea84f1a9510499da5950324: Bootstrap starting.
I20260807 15:11:52.518188 8173 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e5603c6f9ea84f1a9510499da5950324: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 15:11:52.518432 8173 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e5603c6f9ea84f1a9510499da5950324: Bootstrap complete.
I20260807 15:11:52.518668 8173 raft_consensus.cc:348] T 00000000000000000000000000000000 P e5603c6f9ea84f1a9510499da5950324 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "4ebd9481251f4a29ba10f88819ba9ae2" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 39853 } } peers { permanent_uuid: "8e78c2a81ab3476b8cbeff1833835aeb" member_type: VOTER last_known_addr { host: "127.29.194.61" port: 40955 } attrs { promote: false } } peers { permanent_uuid: "e5603c6f9ea84f1a9510499da5950324" member_type: NON_VOTER last_known_addr { host: "127.29.194.60" port: 44209 } attrs { promote: true } }
I20260807 15:11:52.518760 8173 raft_consensus.cc:729] T 00000000000000000000000000000000 P e5603c6f9ea84f1a9510499da5950324 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: e5603c6f9ea84f1a9510499da5950324, State: Initialized, Role: LEARNER
I20260807 15:11:52.518808 8173 consensus_queue.cc:260] T 00000000000000000000000000000000 P e5603c6f9ea84f1a9510499da5950324 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "4ebd9481251f4a29ba10f88819ba9ae2" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 39853 } } peers { permanent_uuid: "8e78c2a81ab3476b8cbeff1833835aeb" member_type: VOTER last_known_addr { host: "127.29.194.61" port: 40955 } attrs { promote: false } } peers { permanent_uuid: "e5603c6f9ea84f1a9510499da5950324" member_type: NON_VOTER last_known_addr { host: "127.29.194.60" port: 44209 } attrs { promote: true } }
I20260807 15:11:52.519096 8173 sys_catalog.cc:564] T 00000000000000000000000000000000 P e5603c6f9ea84f1a9510499da5950324 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:11:52.519045 8176 sys_catalog.cc:455] T 00000000000000000000000000000000 P e5603c6f9ea84f1a9510499da5950324 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "4ebd9481251f4a29ba10f88819ba9ae2" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 39853 } } peers { permanent_uuid: "8e78c2a81ab3476b8cbeff1833835aeb" member_type: VOTER last_known_addr { host: "127.29.194.61" port: 40955 } attrs { promote: false } } peers { permanent_uuid: "e5603c6f9ea84f1a9510499da5950324" member_type: NON_VOTER last_known_addr { host: "127.29.194.60" port: 44209 } attrs { promote: true } } }
I20260807 15:11:52.519269 8176 sys_catalog.cc:458] T 00000000000000000000000000000000 P e5603c6f9ea84f1a9510499da5950324 [sys.catalog]: This master's current role is: LEARNER
I20260807 15:11:52.521162 8192 catalog_manager.cc:1259] Loaded cluster ID: b1e8937b172c41a9914d5edb2b5e84c5
I20260807 15:11:52.521353 8192 catalog_manager.cc:1552] T 00000000000000000000000000000000 P e5603c6f9ea84f1a9510499da5950324: loading cluster ID for follower catalog manager: success
I20260807 15:11:52.522851 8192 catalog_manager.cc:1574] T 00000000000000000000000000000000 P e5603c6f9ea84f1a9510499da5950324: acquiring CA information for follower catalog manager: success
I20260807 15:11:52.523172 8192 catalog_manager.cc:1602] T 00000000000000000000000000000000 P e5603c6f9ea84f1a9510499da5950324: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260807 15:11:52.584311 8073 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:11:52.584625 8073 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:11:52.584672 8073 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:11:52.586534 8073 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:11:52.586591 8073 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:11:52.586637 8073 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.194.1
I20260807 15:11:52.588658 8073 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.194.20:45569
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115480851507-30472-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115480851507-30472-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115480851507-30472-0/krb5kdc/oryx_127.29.194.1.keytab
--principal=oryx/127.29.194.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.194.1:34085
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115480851507-30472-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.194.1
--webserver_port=40935
--tserver_master_addrs=127.29.194.62:39853,127.29.194.61:40955,127.29.194.60:44209
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8073
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.194.1
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115480851507-30472-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:11:52.589169 8073 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:11:52.589545 8073 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:11:52.594949 8199 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:11:52.595161 8200 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:11:52.595361 8202 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:11:52.595798 8073 server_base.cc:1034] running on GCE node
I20260807 15:11:52.596053 8073 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:11:52.596326 8073 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:11:52.597584 8073 hybrid_clock.cc:648] HybridClock initialized: now 1786115512597542 us; error 38 us; skew 500 ppm
Aug 07 15:11:52 dist-test-slave-42z9 krb5kdc[6944](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115512, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.194.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:11:52.600955 8073 init.cc:377] Logged in from keytab as oryx/127.29.194.1@KRBTEST.COM (short username oryx)
I20260807 15:11:52.601507 8073 webserver.cc:460] Webserver started at http://127.29.194.1:40935/ using document root <none> and password file <none>
I20260807 15:11:52.601758 8073 fs_manager.cc:359] Metadata directory not provided
I20260807 15:11:52.601823 8073 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:11:52.603576 8073 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260807 15:11:52.604388 8210 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:11:52.604643 8073 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 15:11:52.604751 8073 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115480851507-30472-0/minicluster-data/ts-0/data,/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115480851507-30472-0/minicluster-data/ts-0/wal
uuid: "10d07cb0cc8840bea5ce1292f24169f3"
format_stamp: "Formatted at 2026-08-07 15:11:51 on dist-test-slave-42z9"
server_key: "b627f490c1cba251174002f94699f33b"
server_key_iv: "023af06e57c7b6422e51248be14a82b7"
server_key_version: "encryptionkey@0"
I20260807 15:11:52.605268 8073 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115480851507-30472-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115480851507-30472-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115480851507-30472-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:11:52.624863 8073 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:11:52.627871 8073 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:11:52.628034 8073 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:11:52.628302 8073 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:11:52.628643 8073 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:11:52.628680 8073 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:11:52.628711 8073 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:11:52.628728 8073 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:11:52.635027 8073 rpc_server.cc:307] RPC server started. Bound to: 127.29.194.1:34085
I20260807 15:11:52.635089 8373 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.194.1:34085 every 8 connection(s)
I20260807 15:11:52.635442 8073 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115480851507-30472-0/minicluster-data/ts-0/data/info.pb
Aug 07 15:11:52 dist-test-slave-42z9 krb5kdc[6944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115512, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.194.1@KRBTEST.COM for oryx/127.29.194.62@KRBTEST.COM
Aug 07 15:11:52 dist-test-slave-42z9 krb5kdc[6944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115512, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.194.1@KRBTEST.COM for oryx/127.29.194.60@KRBTEST.COM
I20260807 15:11:52.644161 30472 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7fZSIL/build/release/bin/kudu as pid 8073
I20260807 15:11:52.644348 30472 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7fZSIL/build/release/bin/kudu with pid 7271
Aug 07 15:11:52 dist-test-slave-42z9 krb5kdc[6944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115512, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.194.1@KRBTEST.COM for oryx/127.29.194.61@KRBTEST.COM
I20260807 15:11:52.652349 8379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:52.636255 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.60:44209 (local address 127.29.194.1:35447)
0807 15:11:52.636461 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0807 15:11:52.636500 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0807 15:11:52.637658 (+ 1158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:11:52.637919 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:11:52.637923 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:11:52.638332 (+ 409us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:11:52.638550 (+ 218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:52.638554 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:52.639758 (+ 1204us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:52.639759 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:52.640506 (+ 747us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:52.640508 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:52.640905 (+ 397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:52.640906 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:52.640981 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:52.641743 (+ 762us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:11:52.641747 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:11:52.644072 (+ 2325us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:11:52.649883 (+ 5811us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:11:52.649903 (+ 20us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:11:52.649907 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:11:52.650373 (+ 466us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:11:52.650652 (+ 279us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:11:52.650653 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:11:52.650654 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:11:52.650830 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:11:52.651186 (+ 356us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:11:52.651195 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:11:52.651449 (+ 254us) client_negotiation.cc:769] Sending connection context
0807 15:11:52.651599 (+ 150us) client_negotiation.cc:240] Negotiation successful
0807 15:11:52.651848 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":110,"threads_started":1}
I20260807 15:11:52.652827 8381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:52.636637 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.194.1:35447 (local address 127.29.194.60:44209)
0807 15:11:52.636791 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:52.636793 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:52.636803 (+ 10us) server_negotiation.cc:407] Connection header received
0807 15:11:52.637763 (+ 960us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:52.637765 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:52.637781 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:52.637842 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:11:52.638646 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:52.639670 (+ 1024us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:52.640575 (+ 905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:52.640831 (+ 256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:52.640896 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:52.644182 (+ 3286us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:11:52.644184 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:11:52.644184 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:11:52.644194 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:11:52.649734 (+ 5540us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:52.650492 (+ 758us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:52.650493 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:52.650494 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:52.650573 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:52.650907 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:52.650908 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:52.650908 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:52.651084 (+ 176us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:11:52.651153 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0807 15:11:52.652609 (+ 1456us) server_negotiation.cc:299] Negotiation successful
0807 15:11:52.652726 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":132,"thread_start_us":74,"threads_started":1}
I20260807 15:11:52.653625 8374 heartbeater.cc:347] Connected to a master server at 127.29.194.60:44209
I20260807 15:11:52.653780 8374 heartbeater.cc:464] Registering TS with master...
I20260807 15:11:52.654088 8374 heartbeater.cc:511] Master 127.29.194.60:44209 requested a full tablet report, sending...
I20260807 15:11:52.654800 8125 ts_manager.cc:194] Registered new tserver with Master: 10d07cb0cc8840bea5ce1292f24169f3 (127.29.194.1:34085)
I20260807 15:11:52.661525 7061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:52.636237 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.194.1:38171 (local address 127.29.194.62:39853)
0807 15:11:52.636300 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:52.636302 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:52.636993 (+ 691us) server_negotiation.cc:407] Connection header received
0807 15:11:52.637788 (+ 795us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:52.637789 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:52.637805 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:52.637835 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:11:52.638670 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:52.639455 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:52.640357 (+ 902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:52.640665 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:52.640735 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:52.643394 (+ 2659us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:11:52.643399 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:11:52.643399 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:11:52.643414 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:11:52.659965 (+ 16551us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:52.660608 (+ 643us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:52.660610 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:52.660610 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:52.660675 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:52.660937 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:52.660939 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:52.660939 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:52.661116 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:11:52.661193 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0807 15:11:52.661307 (+ 114us) server_negotiation.cc:299] Negotiation successful
0807 15:11:52.661403 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260807 15:11:52.661547 7929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:52.636578 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.194.1:58037 (local address 127.29.194.61:40955)
0807 15:11:52.636755 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:52.636757 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:52.636768 (+ 11us) server_negotiation.cc:407] Connection header received
0807 15:11:52.637815 (+ 1047us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:52.637816 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:52.637827 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:52.637869 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:11:52.638651 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:52.639162 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:52.640397 (+ 1235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:52.640689 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:52.640742 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:52.644877 (+ 4135us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:11:52.644879 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:11:52.644879 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:11:52.644888 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:11:52.660123 (+ 15235us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:52.660543 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:52.660544 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:52.660545 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:52.660610 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:52.660890 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:52.660891 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:52.660891 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:52.661059 (+ 168us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:11:52.661123 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0807 15:11:52.661362 (+ 239us) server_negotiation.cc:299] Negotiation successful
0807 15:11:52.661466 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":154}
I20260807 15:11:52.661552 8382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:52.636793 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:39853 (local address 127.29.194.1:38171)
0807 15:11:52.636910 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0807 15:11:52.636919 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0807 15:11:52.637687 (+ 768us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:11:52.637925 (+ 238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:11:52.637925 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:11:52.638332 (+ 407us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:11:52.638550 (+ 218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:52.638557 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:52.639548 (+ 991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:52.639549 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:52.640272 (+ 723us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:52.640275 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:52.640746 (+ 471us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:52.640747 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:52.640884 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:52.641613 (+ 729us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:11:52.641623 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:11:52.643235 (+ 1612us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:11:52.660135 (+ 16900us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:11:52.660137 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:11:52.660137 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:11:52.660485 (+ 348us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:11:52.660738 (+ 253us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:11:52.660739 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:11:52.660739 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:11:52.660841 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:11:52.661178 (+ 337us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:11:52.661178 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:11:52.661212 (+ 34us) client_negotiation.cc:769] Sending connection context
0807 15:11:52.661274 (+ 62us) client_negotiation.cc:240] Negotiation successful
0807 15:11:52.661389 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":102,"mutex_wait_us":6,"thread_start_us":71,"threads_started":1}
I20260807 15:11:52.661839 8380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:52.636250 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.61:40955 (local address 127.29.194.1:58037)
0807 15:11:52.636608 (+ 358us) negotiation.cc:107] Waiting for socket to connect
0807 15:11:52.636618 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 15:11:52.637661 (+ 1043us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:11:52.637924 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:11:52.637925 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:11:52.638329 (+ 404us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:11:52.638551 (+ 222us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:52.638557 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:52.639271 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:52.639272 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:52.640322 (+ 1050us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:52.640324 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:52.640799 (+ 475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:52.640801 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:52.640884 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:52.641688 (+ 804us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:11:52.641695 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:11:52.644777 (+ 3082us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:11:52.660209 (+ 15432us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:11:52.660210 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:11:52.660211 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:11:52.660469 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:11:52.660731 (+ 262us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:11:52.660732 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:11:52.660732 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:11:52.660818 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:11:52.661122 (+ 304us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:11:52.661123 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:11:52.661179 (+ 56us) client_negotiation.cc:769] Sending connection context
0807 15:11:52.661247 (+ 68us) client_negotiation.cc:240] Negotiation successful
0807 15:11:52.661738 (+ 491us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":328,"mutex_wait_us":385,"thread_start_us":112,"threads_started":1}
I20260807 15:11:52.661891 8375 heartbeater.cc:347] Connected to a master server at 127.29.194.62:39853
I20260807 15:11:52.661931 8375 heartbeater.cc:464] Registering TS with master...
I20260807 15:11:52.662014 8375 heartbeater.cc:511] Master 127.29.194.62:39853 requested a full tablet report, sending...
I20260807 15:11:52.662251 8376 heartbeater.cc:347] Connected to a master server at 127.29.194.61:40955
I20260807 15:11:52.662302 8376 heartbeater.cc:464] Registering TS with master...
I20260807 15:11:52.662321 7012 ts_manager.cc:194] Re-registered known tserver with Master: 10d07cb0cc8840bea5ce1292f24169f3 (127.29.194.1:34085)
I20260807 15:11:52.662365 8376 heartbeater.cc:511] Master 127.29.194.61:40955 requested a full tablet report, sending...
I20260807 15:11:52.662714 7859 ts_manager.cc:194] Registered new tserver with Master: 10d07cb0cc8840bea5ce1292f24169f3 (127.29.194.1:34085)
I20260807 15:11:52.662838 7012 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.29.194.1@KRBTEST.COM'} at 127.29.194.1:38171
I20260807 15:11:52.665184 30472 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7fZSIL/build/release/bin/kudu
/tmp/dist-test-task7fZSIL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115480851507-30472-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115480851507-30472-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115480851507-30472-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115480851507-30472-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.194.2:32991
--local_ip_for_outbound_sockets=127.29.194.2
--tserver_master_addrs=127.29.194.62:39853
--webserver_port=44177
--webserver_interface=127.29.194.2
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115480851507-30472-0/krb5kdc/oryx_127.29.194.2.keytab
--principal=oryx/127.29.194.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.194.20:45569
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.29.194.62:39853,127.29.194.61:40955
--tserver_master_addrs=127.29.194.62:39853,127.29.194.61:40955,127.29.194.60:44209 with env {KRB5CCNAME=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115480851507-30472-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115480851507-30472-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115480851507-30472-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:11:52.758770 8383 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:11:52.759018 8383 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:11:52.759045 8383 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:11:52.760401 8383 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:11:52.760456 8383 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:11:52.760485 8383 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.194.2
I20260807 15:11:52.762459 8383 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.194.20:45569
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115480851507-30472-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115480851507-30472-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115480851507-30472-0/krb5kdc/oryx_127.29.194.2.keytab
--principal=oryx/127.29.194.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.194.2:32991
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115480851507-30472-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.194.2
--webserver_port=44177
--tserver_master_addrs=127.29.194.62:39853,127.29.194.61:40955,127.29.194.60:44209
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8383
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.194.2
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115480851507-30472-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:11:52.762724 8383 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:11:52.762969 8383 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:11:52.768605 8399 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:11:52.768612 8394 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:11:52.768605 8396 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:11:52.769078 8383 server_base.cc:1034] running on GCE node
I20260807 15:11:52.769359 8383 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:11:52.769711 8383 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:11:52.770922 8383 hybrid_clock.cc:648] HybridClock initialized: now 1786115512770877 us; error 64 us; skew 500 ppm
Aug 07 15:11:52 dist-test-slave-42z9 krb5kdc[6944](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115512, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.194.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:11:52.774398 8383 init.cc:377] Logged in from keytab as oryx/127.29.194.2@KRBTEST.COM (short username oryx)
I20260807 15:11:52.775087 8383 webserver.cc:460] Webserver started at http://127.29.194.2:44177/ using document root <none> and password file <none>
I20260807 15:11:52.775435 8383 fs_manager.cc:359] Metadata directory not provided
I20260807 15:11:52.775540 8383 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:11:52.777251 8383 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:11:52.778273 8408 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:11:52.778586 8383 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 15:11:52.778748 8383 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115480851507-30472-0/minicluster-data/ts-1/data,/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115480851507-30472-0/minicluster-data/ts-1/wal
uuid: "1a18fd2e4123445cad3e1fa79a7fb96c"
format_stamp: "Formatted at 2026-08-07 15:11:51 on dist-test-slave-42z9"
server_key: "1d6dd3b3c75ea5611c99e5705547ca77"
server_key_iv: "29fbe14ca4cdca03d4b260283090b84d"
server_key_version: "encryptionkey@0"
I20260807 15:11:52.779258 8383 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115480851507-30472-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115480851507-30472-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115480851507-30472-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:11:52.796684 8383 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:11:52.799923 8383 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:11:52.800153 8383 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:11:52.800498 8383 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:11:52.801102 8383 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:11:52.801174 8383 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:11:52.801208 8383 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:11:52.801222 8383 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:11:52.808701 8383 rpc_server.cc:307] RPC server started. Bound to: 127.29.194.2:32991
I20260807 15:11:52.808751 8569 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.194.2:32991 every 8 connection(s)
I20260807 15:11:52.809144 8383 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115480851507-30472-0/minicluster-data/ts-1/data/info.pb
I20260807 15:11:52.812085 30472 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7fZSIL/build/release/bin/kudu as pid 8383
I20260807 15:11:52.812259 30472 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7fZSIL/build/release/bin/kudu with pid 7461
Aug 07 15:11:52 dist-test-slave-42z9 krb5kdc[6944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115512, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.194.2@KRBTEST.COM for oryx/127.29.194.62@KRBTEST.COM
Aug 07 15:11:52 dist-test-slave-42z9 krb5kdc[6944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115512, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.194.2@KRBTEST.COM for oryx/127.29.194.61@KRBTEST.COM
Aug 07 15:11:52 dist-test-slave-42z9 krb5kdc[6944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115512, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.194.2@KRBTEST.COM for oryx/127.29.194.60@KRBTEST.COM
I20260807 15:11:52.822784 7929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:52.810395 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.194.2:54301 (local address 127.29.194.61:40955)
0807 15:11:52.810514 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:52.810517 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:52.810771 (+ 254us) server_negotiation.cc:407] Connection header received
0807 15:11:52.811750 (+ 979us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:52.811751 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:52.811766 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:52.811823 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:11:52.812776 (+ 953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:52.813545 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:52.815102 (+ 1557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:52.815397 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:52.815441 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:52.819268 (+ 3827us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:11:52.819270 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:11:52.819270 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:11:52.819279 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:11:52.821031 (+ 1752us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:52.821630 (+ 599us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:52.821631 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:52.821632 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:52.821693 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:52.821980 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:52.821981 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:52.821982 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:52.822141 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:11:52.822194 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0807 15:11:52.822570 (+ 376us) server_negotiation.cc:299] Negotiation successful
0807 15:11:52.822663 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260807 15:11:52.822839 7061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:52.810199 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.194.2:53097 (local address 127.29.194.62:39853)
0807 15:11:52.810472 (+ 273us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:52.810474 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:52.810492 (+ 18us) server_negotiation.cc:407] Connection header received
0807 15:11:52.811596 (+ 1104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:52.811599 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:52.811617 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:52.811651 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:11:52.813103 (+ 1452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:52.813823 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:52.815091 (+ 1268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:52.815398 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:52.815441 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:52.819230 (+ 3789us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:11:52.819233 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:11:52.819233 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:11:52.819242 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:11:52.821640 (+ 2398us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:52.822064 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:52.822065 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:52.822066 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:52.822120 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:52.822312 (+ 192us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:52.822313 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:52.822313 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:52.822434 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:11:52.822473 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0807 15:11:52.822678 (+ 205us) server_negotiation.cc:299] Negotiation successful
0807 15:11:52.822760 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":224}
I20260807 15:11:52.823036 8381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:52.810284 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.194.2:39829 (local address 127.29.194.60:44209)
0807 15:11:52.810628 (+ 344us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:52.810629 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:52.810637 (+ 8us) server_negotiation.cc:407] Connection header received
0807 15:11:52.811633 (+ 996us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:52.811635 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:52.811668 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:52.811712 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:11:52.815263 (+ 3551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:52.815972 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:52.817399 (+ 1427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:52.817738 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:52.817833 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:52.820005 (+ 2172us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:11:52.820007 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:11:52.820007 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:11:52.820016 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:11:52.821682 (+ 1666us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:52.822129 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:52.822130 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:52.822130 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:52.822189 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:52.822367 (+ 178us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:52.822368 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:52.822369 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:52.822499 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:11:52.822540 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0807 15:11:52.822813 (+ 273us) server_negotiation.cc:299] Negotiation successful
0807 15:11:52.822930 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":320}
I20260807 15:11:52.823421 8576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:52.810057 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.61:40955 (local address 127.29.194.2:54301)
0807 15:11:52.810685 (+ 628us) negotiation.cc:107] Waiting for socket to connect
0807 15:11:52.810693 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0807 15:11:52.811497 (+ 804us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:11:52.811837 (+ 340us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:11:52.811837 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:11:52.812356 (+ 519us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:11:52.812655 (+ 299us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:52.812662 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:52.813679 (+ 1017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:52.813679 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:52.814981 (+ 1302us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:52.814984 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:52.815544 (+ 560us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:52.815545 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:52.815706 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:52.816393 (+ 687us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:11:52.816403 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:11:52.819105 (+ 2702us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:11:52.821144 (+ 2039us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:11:52.821157 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:11:52.821160 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:11:52.821535 (+ 375us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:11:52.821762 (+ 227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:11:52.821762 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:11:52.821763 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:11:52.821899 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:11:52.822225 (+ 326us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:11:52.822233 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:11:52.822415 (+ 182us) client_negotiation.cc:769] Sending connection context
0807 15:11:52.822936 (+ 521us) client_negotiation.cc:240] Negotiation successful
0807 15:11:52.823157 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":599,"mutex_wait_us":255,"thread_start_us":78,"threads_started":1}
I20260807 15:11:52.823421 8575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:52.810057 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:39853 (local address 127.29.194.2:53097)
0807 15:11:52.810264 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0807 15:11:52.810311 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0807 15:11:52.811438 (+ 1127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:11:52.811800 (+ 362us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:11:52.811818 (+ 18us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:11:52.812356 (+ 538us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:11:52.813004 (+ 648us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:52.813006 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:52.813908 (+ 902us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:52.813908 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:52.814981 (+ 1073us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:52.814984 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:52.815551 (+ 567us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:52.815552 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:52.815706 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:52.816471 (+ 765us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:11:52.816481 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:11:52.819105 (+ 2624us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:11:52.821729 (+ 2624us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:11:52.821730 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:11:52.821730 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:11:52.821984 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:11:52.822188 (+ 204us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:11:52.822189 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:11:52.822189 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:11:52.822246 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:11:52.822519 (+ 273us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:11:52.822519 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:11:52.822569 (+ 50us) client_negotiation.cc:769] Sending connection context
0807 15:11:52.822936 (+ 367us) client_negotiation.cc:240] Negotiation successful
0807 15:11:52.823276 (+ 340us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":157,"mutex_wait_us":694,"thread_start_us":109,"threads_started":1}
I20260807 15:11:52.823843 8577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:52.810059 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.60:44209 (local address 127.29.194.2:39829)
0807 15:11:52.810476 (+ 417us) negotiation.cc:107] Waiting for socket to connect
0807 15:11:52.810489 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 15:11:52.811497 (+ 1008us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:11:52.811807 (+ 310us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:11:52.811810 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:11:52.815045 (+ 3235us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:11:52.815131 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:52.815133 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:52.816529 (+ 1396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:52.816530 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:52.817318 (+ 788us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:52.817322 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:52.817844 (+ 522us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:52.817845 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:52.817913 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:52.818367 (+ 454us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:11:52.818373 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:11:52.819894 (+ 1521us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:11:52.821784 (+ 1890us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:11:52.821785 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:11:52.821786 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:11:52.822039 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:11:52.822252 (+ 213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:11:52.822252 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:11:52.822253 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:11:52.822304 (+ 51us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:11:52.822590 (+ 286us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:11:52.822590 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:11:52.822635 (+ 45us) client_negotiation.cc:769] Sending connection context
0807 15:11:52.823163 (+ 528us) client_negotiation.cc:240] Negotiation successful
0807 15:11:52.823343 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":383,"mutex_wait_us":63,"thread_start_us":152,"threads_started":1}
I20260807 15:11:52.824018 8572 heartbeater.cc:347] Connected to a master server at 127.29.194.61:40955
I20260807 15:11:52.824113 8572 heartbeater.cc:464] Registering TS with master...
I20260807 15:11:52.824120 8571 heartbeater.cc:347] Connected to a master server at 127.29.194.62:39853
I20260807 15:11:52.824186 8571 heartbeater.cc:464] Registering TS with master...
I20260807 15:11:52.824249 8570 heartbeater.cc:347] Connected to a master server at 127.29.194.60:44209
I20260807 15:11:52.824290 8570 heartbeater.cc:464] Registering TS with master...
I20260807 15:11:52.824327 8572 heartbeater.cc:511] Master 127.29.194.61:40955 requested a full tablet report, sending...
I20260807 15:11:52.824324 8571 heartbeater.cc:511] Master 127.29.194.62:39853 requested a full tablet report, sending...
I20260807 15:11:52.824333 8570 heartbeater.cc:511] Master 127.29.194.60:44209 requested a full tablet report, sending...
I20260807 15:11:52.824774 7012 ts_manager.cc:194] Re-registered known tserver with Master: 1a18fd2e4123445cad3e1fa79a7fb96c (127.29.194.2:32991)
I20260807 15:11:52.824801 8125 ts_manager.cc:194] Registered new tserver with Master: 1a18fd2e4123445cad3e1fa79a7fb96c (127.29.194.2:32991)
I20260807 15:11:52.824801 7859 ts_manager.cc:194] Registered new tserver with Master: 1a18fd2e4123445cad3e1fa79a7fb96c (127.29.194.2:32991)
I20260807 15:11:52.825099 30472 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7fZSIL/build/release/bin/kudu
/tmp/dist-test-task7fZSIL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115480851507-30472-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115480851507-30472-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115480851507-30472-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115480851507-30472-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.194.3:39783
--local_ip_for_outbound_sockets=127.29.194.3
--tserver_master_addrs=127.29.194.62:39853
--webserver_port=46319
--webserver_interface=127.29.194.3
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115480851507-30472-0/krb5kdc/oryx_127.29.194.3.keytab
--principal=oryx/127.29.194.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.194.20:45569
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.29.194.62:39853,127.29.194.61:40955
--tserver_master_addrs=127.29.194.62:39853,127.29.194.61:40955,127.29.194.60:44209 with env {KRB5CCNAME=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115480851507-30472-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115480851507-30472-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115480851507-30472-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 15:11:52.825249 7012 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.29.194.2@KRBTEST.COM'} at 127.29.194.2:53097
I20260807 15:11:52.905867 8381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:52.902671 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33810 (local address 127.29.194.60:44209)
0807 15:11:52.902849 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:52.902852 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:52.902873 (+ 21us) server_negotiation.cc:407] Connection header received
0807 15:11:52.903018 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:52.903019 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:52.903055 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:52.903143 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:11:52.903299 (+ 156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:52.903953 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:52.904974 (+ 1021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:52.905484 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:52.905551 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:52.905633 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0807 15:11:52.905714 (+ 81us) server_negotiation.cc:299] Negotiation successful
0807 15:11:52.905764 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260807 15:11:52.905867 7801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:52.902559 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.60:44209 (local address 127.0.0.1:33810)
0807 15:11:52.902659 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0807 15:11:52.902672 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 15:11:52.902880 (+ 208us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:11:52.903155 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:11:52.903156 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:11:52.903158 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:11:52.903241 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:52.903243 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:52.904070 (+ 827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:52.904071 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:52.904884 (+ 813us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:52.904887 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:52.905572 (+ 685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:52.905573 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:52.905644 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:52.905650 (+ 6us) client_negotiation.cc:769] Sending connection context
0807 15:11:52.905703 (+ 53us) client_negotiation.cc:240] Negotiation successful
0807 15:11:52.905771 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":72}
I20260807 15:11:52.906526 8138 raft_consensus.cc:1203] T 00000000000000000000000000000000 P e5603c6f9ea84f1a9510499da5950324 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20260807 15:11:52.906728 8176 sys_catalog.cc:455] T 00000000000000000000000000000000 P e5603c6f9ea84f1a9510499da5950324 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4ebd9481251f4a29ba10f88819ba9ae2. Latest consensus state: current_term: 1 leader_uuid: "4ebd9481251f4a29ba10f88819ba9ae2" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "4ebd9481251f4a29ba10f88819ba9ae2" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 39853 } } peers { permanent_uuid: "8e78c2a81ab3476b8cbeff1833835aeb" member_type: VOTER last_known_addr { host: "127.29.194.61" port: 40955 } attrs { promote: false } } peers { permanent_uuid: "e5603c6f9ea84f1a9510499da5950324" member_type: NON_VOTER last_known_addr { host: "127.29.194.60" port: 44209 } attrs { promote: true } } }
I20260807 15:11:52.906842 8176 sys_catalog.cc:458] T 00000000000000000000000000000000 P e5603c6f9ea84f1a9510499da5950324 [sys.catalog]: This master's current role is: LEARNER
W20260807 15:11:52.915055 8578 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:11:52.915294 8578 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:11:52.915320 8578 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:11:52.916543 8578 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:11:52.916586 8578 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:11:52.916615 8578 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.194.3
I20260807 15:11:52.918212 8578 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.194.20:45569
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115480851507-30472-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115480851507-30472-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115480851507-30472-0/krb5kdc/oryx_127.29.194.3.keytab
--principal=oryx/127.29.194.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.194.3:39783
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115480851507-30472-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.194.3
--webserver_port=46319
--tserver_master_addrs=127.29.194.62:39853,127.29.194.61:40955,127.29.194.60:44209
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8578
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.194.3
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115480851507-30472-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:11:52.918459 8578 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:11:52.918709 8578 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:11:52.924432 8584 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:11:52.924439 8585 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:11:52.924594 8578 server_base.cc:1034] running on GCE node
W20260807 15:11:52.924719 8587 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:11:52.925295 8578 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:11:52.925685 8578 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:11:52.926874 8578 hybrid_clock.cc:648] HybridClock initialized: now 1786115512926856 us; error 42 us; skew 500 ppm
Aug 07 15:11:52 dist-test-slave-42z9 krb5kdc[6944](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115512, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.194.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:11:52.929880 8578 init.cc:377] Logged in from keytab as oryx/127.29.194.3@KRBTEST.COM (short username oryx)
I20260807 15:11:52.930397 8578 webserver.cc:460] Webserver started at http://127.29.194.3:46319/ using document root <none> and password file <none>
I20260807 15:11:52.930644 8578 fs_manager.cc:359] Metadata directory not provided
I20260807 15:11:52.930713 8578 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:11:52.932197 8578 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:11:52.932843 8595 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:11:52.933090 8578 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:11:52.933190 8578 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115480851507-30472-0/minicluster-data/ts-2/data,/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115480851507-30472-0/minicluster-data/ts-2/wal
uuid: "9d215d5c47fd4619ad846c7151ad85c1"
format_stamp: "Formatted at 2026-08-07 15:11:51 on dist-test-slave-42z9"
server_key: "74ab483c6f40a59242524869d261d3e6"
server_key_iv: "4988e34e950d3782d32a8affa81de027"
server_key_version: "encryptionkey@0"
I20260807 15:11:52.933601 8578 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115480851507-30472-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115480851507-30472-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115480851507-30472-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:11:52.944049 8578 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:11:52.947304 8578 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:11:52.947481 8578 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:11:52.947722 8578 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:11:52.948055 8578 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:11:52.948089 8578 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:11:52.948122 8578 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:11:52.948145 8578 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:11:52.954336 8578 rpc_server.cc:307] RPC server started. Bound to: 127.29.194.3:39783
I20260807 15:11:52.954387 8755 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.194.3:39783 every 8 connection(s)
I20260807 15:11:52.954766 8578 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115480851507-30472-0/minicluster-data/ts-2/data/info.pb
I20260807 15:11:52.962261 30472 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7fZSIL/build/release/bin/kudu as pid 8578
Aug 07 15:11:52 dist-test-slave-42z9 krb5kdc[6944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115512, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.194.3@KRBTEST.COM for oryx/127.29.194.62@KRBTEST.COM
Aug 07 15:11:52 dist-test-slave-42z9 krb5kdc[6944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115512, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.194.3@KRBTEST.COM for oryx/127.29.194.61@KRBTEST.COM
Aug 07 15:11:52 dist-test-slave-42z9 krb5kdc[6944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115512, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.194.3@KRBTEST.COM for oryx/127.29.194.60@KRBTEST.COM
I20260807 15:11:52.970957 7061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:52.955639 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.194.3:50247 (local address 127.29.194.62:39853)
0807 15:11:52.955697 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:52.955700 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:52.955947 (+ 247us) server_negotiation.cc:407] Connection header received
0807 15:11:52.956604 (+ 657us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:52.956605 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:52.956620 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:52.956668 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:11:52.958403 (+ 1735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:52.958893 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:52.960305 (+ 1412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:52.960650 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:52.960741 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:52.964950 (+ 4209us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:11:52.964951 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:11:52.964952 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:11:52.964960 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:11:52.968063 (+ 3103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:52.969711 (+ 1648us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:52.969712 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:52.969713 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:52.969774 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:52.970137 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:52.970138 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:52.970138 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:52.970281 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:11:52.970324 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0807 15:11:52.970763 (+ 439us) server_negotiation.cc:299] Negotiation successful
0807 15:11:52.970853 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260807 15:11:52.971542 8764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:52.955526 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:39853 (local address 127.29.194.3:50247)
0807 15:11:52.955895 (+ 369us) negotiation.cc:107] Waiting for socket to connect
0807 15:11:52.955903 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0807 15:11:52.956454 (+ 551us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:11:52.957054 (+ 600us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:11:52.957055 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:11:52.957213 (+ 158us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:11:52.957344 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:52.957350 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:52.958968 (+ 1618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:52.958970 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:52.960142 (+ 1172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:52.960146 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:52.960768 (+ 622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:52.960769 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:52.960923 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:52.961767 (+ 844us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:11:52.961778 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:11:52.964853 (+ 3075us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:11:52.968411 (+ 3558us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:11:52.968422 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:11:52.968424 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:11:52.968785 (+ 361us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:11:52.969859 (+ 1074us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:11:52.969860 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:11:52.969860 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:11:52.970050 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:11:52.970380 (+ 330us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:11:52.970387 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:11:52.970585 (+ 198us) client_negotiation.cc:769] Sending connection context
0807 15:11:52.971079 (+ 494us) client_negotiation.cc:240] Negotiation successful
0807 15:11:52.971288 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":342,"mutex_wait_us":41,"thread_start_us":111,"threads_started":1}
I20260807 15:11:52.972024 8757 heartbeater.cc:347] Connected to a master server at 127.29.194.62:39853
I20260807 15:11:52.972134 8757 heartbeater.cc:464] Registering TS with master...
I20260807 15:11:52.972260 8763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:52.955526 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.61:40955 (local address 127.29.194.3:44613)
0807 15:11:52.955786 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0807 15:11:52.955848 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0807 15:11:52.956451 (+ 603us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:11:52.956704 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:11:52.956705 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:11:52.957119 (+ 414us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:11:52.957344 (+ 225us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:52.957350 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:52.958491 (+ 1141us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:52.958493 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:52.960142 (+ 1649us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:52.960146 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:52.961273 (+ 1127us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:52.961274 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:52.961369 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:52.961886 (+ 517us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:11:52.961914 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:11:52.964287 (+ 2373us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:11:52.970514 (+ 6227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:11:52.970515 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:11:52.970516 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:11:52.970735 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:11:52.970956 (+ 221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:11:52.970957 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:11:52.970957 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:11:52.971019 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:11:52.971935 (+ 916us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:11:52.971936 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:11:52.971985 (+ 49us) client_negotiation.cc:769] Sending connection context
0807 15:11:52.972047 (+ 62us) client_negotiation.cc:240] Negotiation successful
0807 15:11:52.972157 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":212,"mutex_wait_us":33,"thread_start_us":256,"threads_started":1}
I20260807 15:11:52.972339 8757 heartbeater.cc:511] Master 127.29.194.62:39853 requested a full tablet report, sending...
I20260807 15:11:52.972448 8771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:52.963728 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43364 (local address 127.29.194.62:39853)
0807 15:11:52.964787 (+ 1059us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:52.964789 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:52.964799 (+ 10us) server_negotiation.cc:407] Connection header received
0807 15:11:52.964829 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:52.964830 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:52.964845 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:52.964919 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:11:52.965790 (+ 871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:52.966596 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:52.967691 (+ 1095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:52.968001 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:52.968058 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:52.969468 (+ 1410us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:11:52.969470 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:11:52.969470 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:11:52.969478 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:11:52.971087 (+ 1609us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:52.971446 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:52.971447 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:52.971447 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:52.971518 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:52.971721 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:52.971721 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:52.971722 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:52.971845 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:11:52.971936 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0807 15:11:52.972285 (+ 349us) server_negotiation.cc:299] Negotiation successful
0807 15:11:52.972369 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1027,"thread_start_us":104,"threads_started":1}
I20260807 15:11:52.972981 7012 ts_manager.cc:194] Re-registered known tserver with Master: 9d215d5c47fd4619ad846c7151ad85c1 (127.29.194.3:39783)
I20260807 15:11:52.973356 7929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:52.955712 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.194.3:44613 (local address 127.29.194.61:40955)
0807 15:11:52.955759 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:52.955760 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:52.955938 (+ 178us) server_negotiation.cc:407] Connection header received
0807 15:11:52.956575 (+ 637us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:52.956576 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:52.956589 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:52.956638 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:11:52.957497 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:52.958291 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:52.960834 (+ 2543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:52.961126 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:52.961198 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:52.964463 (+ 3265us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:11:52.964465 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:11:52.964466 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:11:52.964476 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:11:52.970428 (+ 5952us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:52.970817 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:52.970818 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:52.970819 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:52.970884 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:52.971503 (+ 619us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:52.971504 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:52.971504 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:52.971643 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:11:52.971708 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0807 15:11:52.973182 (+ 1474us) server_negotiation.cc:299] Negotiation successful
0807 15:11:52.973270 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260807 15:11:52.973472 7012 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.29.194.3@KRBTEST.COM'} at 127.29.194.3:50247
I20260807 15:11:52.973693 8761 heartbeater.cc:347] Connected to a master server at 127.29.194.61:40955
I20260807 15:11:52.973748 8761 heartbeater.cc:464] Registering TS with master...
I20260807 15:11:52.973820 8761 heartbeater.cc:511] Master 127.29.194.61:40955 requested a full tablet report, sending...
I20260807 15:11:52.974054 8773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:52.963946 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33818 (local address 127.29.194.60:44209)
0807 15:11:52.964437 (+ 491us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:52.964439 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:52.964597 (+ 158us) server_negotiation.cc:407] Connection header received
0807 15:11:52.964620 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:52.964621 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:52.964635 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:52.965052 (+ 417us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:11:52.966030 (+ 978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:52.966765 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:52.967715 (+ 950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:52.968022 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:52.968072 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:52.970376 (+ 2304us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:11:52.970378 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:11:52.970378 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:11:52.970386 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:11:52.971986 (+ 1600us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:52.972765 (+ 779us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:52.972767 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:52.972767 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:52.972825 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:52.973028 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:52.973029 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:52.973029 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:52.973151 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:11:52.973493 (+ 342us) server_negotiation.cc:1035] Waiting for connection context
0807 15:11:52.973872 (+ 379us) server_negotiation.cc:299] Negotiation successful
0807 15:11:52.973965 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":466,"thread_start_us":70,"threads_started":1}
I20260807 15:11:52.974115 7859 ts_manager.cc:194] Registered new tserver with Master: 9d215d5c47fd4619ad846c7151ad85c1 (127.29.194.3:39783)
I20260807 15:11:52.975076 8381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:52.955821 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.194.3:47881 (local address 127.29.194.60:44209)
0807 15:11:52.955962 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:52.955965 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:52.955974 (+ 9us) server_negotiation.cc:407] Connection header received
0807 15:11:52.956518 (+ 544us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:52.956520 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:52.956545 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:52.956597 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:11:52.957467 (+ 870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:52.958291 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:52.960926 (+ 2635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:52.961293 (+ 367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:52.961371 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:52.964718 (+ 3347us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:11:52.964719 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:11:52.964719 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:11:52.964727 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:11:52.970102 (+ 5375us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:52.972990 (+ 2888us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:52.972991 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:52.972991 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:52.973045 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:52.974457 (+ 1412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:52.974458 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:52.974460 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:52.974593 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:11:52.974662 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0807 15:11:52.974824 (+ 162us) server_negotiation.cc:299] Negotiation successful
0807 15:11:52.974991 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":120}
I20260807 15:11:52.975170 8765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:52.955526 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.60:44209 (local address 127.29.194.3:47881)
0807 15:11:52.955852 (+ 326us) negotiation.cc:107] Waiting for socket to connect
0807 15:11:52.955863 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 15:11:52.956414 (+ 551us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:11:52.956691 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:11:52.956696 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:11:52.957124 (+ 428us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:11:52.957344 (+ 220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:52.957348 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:52.958491 (+ 1143us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:52.958493 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:52.960812 (+ 2319us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:52.960814 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:52.961380 (+ 566us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:52.961381 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:52.961537 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:52.962262 (+ 725us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:11:52.962267 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:11:52.964636 (+ 2369us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:11:52.972713 (+ 8077us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:11:52.972714 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:11:52.972715 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:11:52.972916 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:11:52.974306 (+ 1390us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:11:52.974307 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:11:52.974308 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:11:52.974387 (+ 79us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:11:52.974658 (+ 271us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:11:52.974658 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:11:52.974703 (+ 45us) client_negotiation.cc:769] Sending connection context
0807 15:11:52.974781 (+ 78us) client_negotiation.cc:240] Negotiation successful
0807 15:11:52.974895 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":289,"mutex_wait_us":1357,"thread_start_us":107,"threads_started":1}
I20260807 15:11:52.975515 8756 heartbeater.cc:347] Connected to a master server at 127.29.194.60:44209
I20260807 15:11:52.975589 8756 heartbeater.cc:464] Registering TS with master...
I20260807 15:11:52.975674 8756 heartbeater.cc:511] Master 127.29.194.60:44209 requested a full tablet report, sending...
I20260807 15:11:52.975973 8125 ts_manager.cc:194] Registered new tserver with Master: 9d215d5c47fd4619ad846c7151ad85c1 (127.29.194.3:39783)
I20260807 15:11:52.974472 7012 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43364:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 15:11:52.976832 7012 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 15:11:52.977183 8777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:52.965812 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51710 (local address 127.29.194.61:40955)
0807 15:11:52.965994 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:52.965996 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:52.966005 (+ 9us) server_negotiation.cc:407] Connection header received
0807 15:11:52.966033 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:52.966034 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:52.966046 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:52.966778 (+ 732us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:11:52.966822 (+ 44us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:52.967564 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:52.968602 (+ 1038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:52.968920 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:52.969241 (+ 321us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:52.972195 (+ 2954us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:11:52.972196 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:11:52.972196 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:11:52.972203 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:11:52.976087 (+ 3884us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:52.976417 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:52.976418 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:52.976418 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:52.976468 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:52.976659 (+ 191us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:52.976660 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:52.976661 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:52.976790 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:11:52.976877 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0807 15:11:52.977003 (+ 126us) server_negotiation.cc:299] Negotiation successful
0807 15:11:52.977099 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":158,"thread_start_us":118,"threads_started":1}
I20260807 15:11:52.977710 8074 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4ebd9481251f4a29ba10f88819ba9ae2 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "4ebd9481251f4a29ba10f88819ba9ae2" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "4ebd9481251f4a29ba10f88819ba9ae2" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 39853 } } peers { permanent_uuid: "8e78c2a81ab3476b8cbeff1833835aeb" member_type: VOTER last_known_addr { host: "127.29.194.61" port: 40955 } attrs { promote: false } } peers { permanent_uuid: "e5603c6f9ea84f1a9510499da5950324" member_type: NON_VOTER last_known_addr { host: "127.29.194.60" port: 44209 } attrs { promote: true } } }
I20260807 15:11:52.977811 8074 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4ebd9481251f4a29ba10f88819ba9ae2 [sys.catalog]: This master's current role is: LEADER
I20260807 15:11:52.978874 8779 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 4ebd9481251f4a29ba10f88819ba9ae2: attempting to promote NON_VOTER e5603c6f9ea84f1a9510499da5950324 to VOTER
I20260807 15:11:52.979007 8779 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4ebd9481251f4a29ba10f88819ba9ae2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "4ebd9481251f4a29ba10f88819ba9ae2" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 39853 } } peers { permanent_uuid: "8e78c2a81ab3476b8cbeff1833835aeb" member_type: VOTER last_known_addr { host: "127.29.194.61" port: 40955 } attrs { promote: false } } peers { permanent_uuid: "e5603c6f9ea84f1a9510499da5950324" member_type: VOTER last_known_addr { host: "127.29.194.60" port: 44209 } attrs { promote: false } }
I20260807 15:11:52.979414 8138 raft_consensus.cc:1261] T 00000000000000000000000000000000 P e5603c6f9ea84f1a9510499da5950324 [term 1 LEARNER]: Refusing update from remote peer 4ebd9481251f4a29ba10f88819ba9ae2: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260807 15:11:52.979542 7868 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 8e78c2a81ab3476b8cbeff1833835aeb [term 1 FOLLOWER]: Refusing update from remote peer 4ebd9481251f4a29ba10f88819ba9ae2: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260807 15:11:52.979640 8780 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 4ebd9481251f4a29ba10f88819ba9ae2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5603c6f9ea84f1a9510499da5950324" member_type: VOTER last_known_addr { host: "127.29.194.60" port: 44209 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260807 15:11:52.979728 8074 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 4ebd9481251f4a29ba10f88819ba9ae2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e78c2a81ab3476b8cbeff1833835aeb" member_type: VOTER last_known_addr { host: "127.29.194.61" port: 40955 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260807 15:11:52.980327 8074 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 4ebd9481251f4a29ba10f88819ba9ae2 [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 7 to 9, e5603c6f9ea84f1a9510499da5950324 (127.29.194.60) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "4ebd9481251f4a29ba10f88819ba9ae2" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 39853 } } peers { permanent_uuid: "8e78c2a81ab3476b8cbeff1833835aeb" member_type: VOTER last_known_addr { host: "127.29.194.61" port: 40955 } attrs { promote: false } } peers { permanent_uuid: "e5603c6f9ea84f1a9510499da5950324" member_type: VOTER last_known_addr { host: "127.29.194.60" port: 44209 } attrs { promote: false } } }
I20260807 15:11:52.980485 7868 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 8e78c2a81ab3476b8cbeff1833835aeb [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, e5603c6f9ea84f1a9510499da5950324 (127.29.194.60) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "4ebd9481251f4a29ba10f88819ba9ae2" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 39853 } } peers { permanent_uuid: "8e78c2a81ab3476b8cbeff1833835aeb" member_type: VOTER last_known_addr { host: "127.29.194.61" port: 40955 } attrs { promote: false } } peers { permanent_uuid: "e5603c6f9ea84f1a9510499da5950324" member_type: VOTER last_known_addr { host: "127.29.194.60" port: 44209 } attrs { promote: false } } }
I20260807 15:11:52.980677 8138 raft_consensus.cc:2932] T 00000000000000000000000000000000 P e5603c6f9ea84f1a9510499da5950324 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, e5603c6f9ea84f1a9510499da5950324 (127.29.194.60) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "4ebd9481251f4a29ba10f88819ba9ae2" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 39853 } } peers { permanent_uuid: "8e78c2a81ab3476b8cbeff1833835aeb" member_type: VOTER last_known_addr { host: "127.29.194.61" port: 40955 } attrs { promote: false } } peers { permanent_uuid: "e5603c6f9ea84f1a9510499da5950324" member_type: VOTER last_known_addr { host: "127.29.194.60" port: 44209 } attrs { promote: false } } }
I20260807 15:11:52.981307 8074 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4ebd9481251f4a29ba10f88819ba9ae2 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "4ebd9481251f4a29ba10f88819ba9ae2" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "4ebd9481251f4a29ba10f88819ba9ae2" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 39853 } } peers { permanent_uuid: "8e78c2a81ab3476b8cbeff1833835aeb" member_type: VOTER last_known_addr { host: "127.29.194.61" port: 40955 } attrs { promote: false } } peers { permanent_uuid: "e5603c6f9ea84f1a9510499da5950324" member_type: VOTER last_known_addr { host: "127.29.194.60" port: 44209 } attrs { promote: false } } }
I20260807 15:11:52.981377 8074 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4ebd9481251f4a29ba10f88819ba9ae2 [sys.catalog]: This master's current role is: LEADER
I20260807 15:11:52.981428 8074 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4ebd9481251f4a29ba10f88819ba9ae2 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "4ebd9481251f4a29ba10f88819ba9ae2" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "4ebd9481251f4a29ba10f88819ba9ae2" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 39853 } } peers { permanent_uuid: "8e78c2a81ab3476b8cbeff1833835aeb" member_type: VOTER last_known_addr { host: "127.29.194.61" port: 40955 } attrs { promote: false } } peers { permanent_uuid: "e5603c6f9ea84f1a9510499da5950324" member_type: VOTER last_known_addr { host: "127.29.194.60" port: 44209 } attrs { promote: false } } }
I20260807 15:11:52.981490 8074 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4ebd9481251f4a29ba10f88819ba9ae2 [sys.catalog]: This master's current role is: LEADER
I20260807 15:11:52.981528 8074 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4ebd9481251f4a29ba10f88819ba9ae2 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "4ebd9481251f4a29ba10f88819ba9ae2" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "4ebd9481251f4a29ba10f88819ba9ae2" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 39853 } } peers { permanent_uuid: "8e78c2a81ab3476b8cbeff1833835aeb" member_type: VOTER last_known_addr { host: "127.29.194.61" port: 40955 } attrs { promote: false } } peers { permanent_uuid: "e5603c6f9ea84f1a9510499da5950324" member_type: VOTER last_known_addr { host: "127.29.194.60" port: 44209 } attrs { promote: false } } }
I20260807 15:11:52.981570 8074 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4ebd9481251f4a29ba10f88819ba9ae2 [sys.catalog]: This master's current role is: LEADER
I20260807 15:11:52.981885 8790 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8e78c2a81ab3476b8cbeff1833835aeb [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "4ebd9481251f4a29ba10f88819ba9ae2" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "4ebd9481251f4a29ba10f88819ba9ae2" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 39853 } } peers { permanent_uuid: "8e78c2a81ab3476b8cbeff1833835aeb" member_type: VOTER last_known_addr { host: "127.29.194.61" port: 40955 } attrs { promote: false } } peers { permanent_uuid: "e5603c6f9ea84f1a9510499da5950324" member_type: VOTER last_known_addr { host: "127.29.194.60" port: 44209 } attrs { promote: false } } }
I20260807 15:11:52.981968 8790 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8e78c2a81ab3476b8cbeff1833835aeb [sys.catalog]: This master's current role is: FOLLOWER
I20260807 15:11:52.982252 8176 sys_catalog.cc:455] T 00000000000000000000000000000000 P e5603c6f9ea84f1a9510499da5950324 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "4ebd9481251f4a29ba10f88819ba9ae2" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "4ebd9481251f4a29ba10f88819ba9ae2" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 39853 } } peers { permanent_uuid: "8e78c2a81ab3476b8cbeff1833835aeb" member_type: VOTER last_known_addr { host: "127.29.194.61" port: 40955 } attrs { promote: false } } peers { permanent_uuid: "e5603c6f9ea84f1a9510499da5950324" member_type: VOTER last_known_addr { host: "127.29.194.60" port: 44209 } attrs { promote: false } } }
I20260807 15:11:52.982447 8176 sys_catalog.cc:458] T 00000000000000000000000000000000 P e5603c6f9ea84f1a9510499da5950324 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 15:11:52.988765 7801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:52.983906 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.3:39783 (local address 127.0.0.1:47964)
0807 15:11:52.984075 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0807 15:11:52.984090 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 15:11:52.984139 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:11:52.984692 (+ 553us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:11:52.984693 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:11:52.984695 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:11:52.984800 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:52.984803 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:52.986426 (+ 1623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:52.986427 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:52.987418 (+ 991us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:52.987422 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:52.988475 (+ 1053us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:52.988476 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:52.988543 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:52.988548 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 15:11:52.988616 (+ 68us) client_negotiation.cc:240] Negotiation successful
0807 15:11:52.988672 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"mutex_wait_us":86}
I20260807 15:11:52.988886 8796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:52.984602 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38680 (local address 127.29.194.1:34085)
0807 15:11:52.985005 (+ 403us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:52.985011 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:52.985032 (+ 21us) server_negotiation.cc:407] Connection header received
0807 15:11:52.985074 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:52.985076 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:52.985093 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:52.985163 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:11:52.985270 (+ 107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:52.986303 (+ 1033us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:52.988001 (+ 1698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:52.988546 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:52.988616 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:52.988685 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0807 15:11:52.988767 (+ 82us) server_negotiation.cc:299] Negotiation successful
0807 15:11:52.988811 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":363,"thread_start_us":105,"threads_started":1}
I20260807 15:11:52.988876 8794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:52.984205 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.1:34085 (local address 127.0.0.1:38680)
0807 15:11:52.984319 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0807 15:11:52.984328 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0807 15:11:52.984366 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:11:52.985174 (+ 808us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:11:52.985175 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:11:52.985176 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:11:52.985219 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:52.985221 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:52.986405 (+ 1184us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:52.986406 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:52.987921 (+ 1515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:52.987924 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:52.988630 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:52.988631 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:52.988695 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:52.988700 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 15:11:52.988754 (+ 54us) client_negotiation.cc:240] Negotiation successful
0807 15:11:52.988799 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":95,"mutex_wait_us":36,"thread_start_us":80,"threads_started":1}
I20260807 15:11:52.989192 8795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:52.984241 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47964 (local address 127.29.194.3:39783)
0807 15:11:52.984467 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:52.984475 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:52.984487 (+ 12us) server_negotiation.cc:407] Connection header received
0807 15:11:52.984542 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:52.984545 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:52.984566 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:52.984896 (+ 330us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:11:52.984954 (+ 58us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:52.986358 (+ 1404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:52.987563 (+ 1205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:52.988368 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:52.988461 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:52.988555 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0807 15:11:52.989052 (+ 497us) server_negotiation.cc:299] Negotiation successful
0807 15:11:52.989109 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":181,"thread_start_us":135,"threads_started":1}
I20260807 15:11:52.989555 8791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:52.983960 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.2:32991 (local address 127.0.0.1:39864)
0807 15:11:52.984140 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0807 15:11:52.984152 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 15:11:52.984201 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:11:52.984842 (+ 641us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:11:52.984843 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:11:52.984846 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:11:52.984921 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:52.984924 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:52.986386 (+ 1462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:52.986387 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:52.987657 (+ 1270us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:52.987660 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:52.988734 (+ 1074us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:52.988735 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:52.988789 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:52.988793 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 15:11:52.989375 (+ 582us) client_negotiation.cc:240] Negotiation successful
0807 15:11:52.989429 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152,"mutex_wait_us":104,"thread_start_us":133,"threads_started":1}
I20260807 15:11:52.989874 8792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:52.984090 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39864 (local address 127.29.194.2:32991)
0807 15:11:52.984636 (+ 546us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:52.984640 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:52.984651 (+ 11us) server_negotiation.cc:407] Connection header received
0807 15:11:52.984694 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:52.984696 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:52.984710 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:52.984827 (+ 117us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:11:52.985019 (+ 192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:52.986257 (+ 1238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:52.987767 (+ 1510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:52.988563 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:52.988633 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:52.988701 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0807 15:11:52.989739 (+ 1038us) server_negotiation.cc:299] Negotiation successful
0807 15:11:52.989799 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":508,"thread_start_us":133,"threads_started":1}
I20260807 15:11:52.989758 8278 tablet_service.cc:1467] Processing CreateTablet for tablet 1c90d746d96b492fad569026a4cebe61 (DEFAULT_TABLE table=test-table [id=788b40fc7bcd43d8878e300e62b4e5b3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:11:52.989902 8666 tablet_service.cc:1467] Processing CreateTablet for tablet 1c90d746d96b492fad569026a4cebe61 (DEFAULT_TABLE table=test-table [id=788b40fc7bcd43d8878e300e62b4e5b3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:11:52.990209 8278 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1c90d746d96b492fad569026a4cebe61. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:11:52.990255 8666 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1c90d746d96b492fad569026a4cebe61. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:11:52.990450 8473 tablet_service.cc:1467] Processing CreateTablet for tablet 1c90d746d96b492fad569026a4cebe61 (DEFAULT_TABLE table=test-table [id=788b40fc7bcd43d8878e300e62b4e5b3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:11:52.990818 8473 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1c90d746d96b492fad569026a4cebe61. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:11:52.993688 8797 tablet_bootstrap.cc:492] T 1c90d746d96b492fad569026a4cebe61 P 9d215d5c47fd4619ad846c7151ad85c1: Bootstrap starting.
I20260807 15:11:52.993790 8799 tablet_bootstrap.cc:492] T 1c90d746d96b492fad569026a4cebe61 P 10d07cb0cc8840bea5ce1292f24169f3: Bootstrap starting.
I20260807 15:11:52.994395 8803 tablet_bootstrap.cc:492] T 1c90d746d96b492fad569026a4cebe61 P 1a18fd2e4123445cad3e1fa79a7fb96c: Bootstrap starting.
I20260807 15:11:52.994545 8799 tablet_bootstrap.cc:654] T 1c90d746d96b492fad569026a4cebe61 P 10d07cb0cc8840bea5ce1292f24169f3: Neither blocks nor log segments found. Creating new log.
I20260807 15:11:52.994566 8797 tablet_bootstrap.cc:654] T 1c90d746d96b492fad569026a4cebe61 P 9d215d5c47fd4619ad846c7151ad85c1: Neither blocks nor log segments found. Creating new log.
I20260807 15:11:52.994913 8797 log.cc:824] T 1c90d746d96b492fad569026a4cebe61 P 9d215d5c47fd4619ad846c7151ad85c1: Log is configured to *not* fsync() on all Append() calls
I20260807 15:11:52.994904 8799 log.cc:824] T 1c90d746d96b492fad569026a4cebe61 P 10d07cb0cc8840bea5ce1292f24169f3: Log is configured to *not* fsync() on all Append() calls
I20260807 15:11:52.994989 8803 tablet_bootstrap.cc:654] T 1c90d746d96b492fad569026a4cebe61 P 1a18fd2e4123445cad3e1fa79a7fb96c: Neither blocks nor log segments found. Creating new log.
I20260807 15:11:52.995322 8803 log.cc:824] T 1c90d746d96b492fad569026a4cebe61 P 1a18fd2e4123445cad3e1fa79a7fb96c: Log is configured to *not* fsync() on all Append() calls
I20260807 15:11:52.995852 8797 tablet_bootstrap.cc:492] T 1c90d746d96b492fad569026a4cebe61 P 9d215d5c47fd4619ad846c7151ad85c1: No bootstrap required, opened a new log
I20260807 15:11:52.995934 8797 ts_tablet_manager.cc:1397] T 1c90d746d96b492fad569026a4cebe61 P 9d215d5c47fd4619ad846c7151ad85c1: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 15:11:52.995998 8803 tablet_bootstrap.cc:492] T 1c90d746d96b492fad569026a4cebe61 P 1a18fd2e4123445cad3e1fa79a7fb96c: No bootstrap required, opened a new log
I20260807 15:11:52.996006 8799 tablet_bootstrap.cc:492] T 1c90d746d96b492fad569026a4cebe61 P 10d07cb0cc8840bea5ce1292f24169f3: No bootstrap required, opened a new log
I20260807 15:11:52.996064 8803 ts_tablet_manager.cc:1397] T 1c90d746d96b492fad569026a4cebe61 P 1a18fd2e4123445cad3e1fa79a7fb96c: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 15:11:52.996078 8799 ts_tablet_manager.cc:1397] T 1c90d746d96b492fad569026a4cebe61 P 10d07cb0cc8840bea5ce1292f24169f3: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 15:11:52.998041 8803 raft_consensus.cc:348] T 1c90d746d96b492fad569026a4cebe61 P 1a18fd2e4123445cad3e1fa79a7fb96c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d215d5c47fd4619ad846c7151ad85c1" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 39783 } } peers { permanent_uuid: "1a18fd2e4123445cad3e1fa79a7fb96c" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 32991 } } peers { permanent_uuid: "10d07cb0cc8840bea5ce1292f24169f3" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 34085 } }
I20260807 15:11:52.998200 8803 raft_consensus.cc:374] T 1c90d746d96b492fad569026a4cebe61 P 1a18fd2e4123445cad3e1fa79a7fb96c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:11:52.998091 8799 raft_consensus.cc:348] T 1c90d746d96b492fad569026a4cebe61 P 10d07cb0cc8840bea5ce1292f24169f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d215d5c47fd4619ad846c7151ad85c1" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 39783 } } peers { permanent_uuid: "1a18fd2e4123445cad3e1fa79a7fb96c" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 32991 } } peers { permanent_uuid: "10d07cb0cc8840bea5ce1292f24169f3" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 34085 } }
I20260807 15:11:52.998234 8803 raft_consensus.cc:729] T 1c90d746d96b492fad569026a4cebe61 P 1a18fd2e4123445cad3e1fa79a7fb96c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1a18fd2e4123445cad3e1fa79a7fb96c, State: Initialized, Role: FOLLOWER
I20260807 15:11:52.998270 8799 raft_consensus.cc:374] T 1c90d746d96b492fad569026a4cebe61 P 10d07cb0cc8840bea5ce1292f24169f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:11:52.998301 8799 raft_consensus.cc:729] T 1c90d746d96b492fad569026a4cebe61 P 10d07cb0cc8840bea5ce1292f24169f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 10d07cb0cc8840bea5ce1292f24169f3, State: Initialized, Role: FOLLOWER
I20260807 15:11:52.998374 8803 consensus_queue.cc:260] T 1c90d746d96b492fad569026a4cebe61 P 1a18fd2e4123445cad3e1fa79a7fb96c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d215d5c47fd4619ad846c7151ad85c1" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 39783 } } peers { permanent_uuid: "1a18fd2e4123445cad3e1fa79a7fb96c" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 32991 } } peers { permanent_uuid: "10d07cb0cc8840bea5ce1292f24169f3" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 34085 } }
I20260807 15:11:52.998433 8799 consensus_queue.cc:260] T 1c90d746d96b492fad569026a4cebe61 P 10d07cb0cc8840bea5ce1292f24169f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d215d5c47fd4619ad846c7151ad85c1" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 39783 } } peers { permanent_uuid: "1a18fd2e4123445cad3e1fa79a7fb96c" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 32991 } } peers { permanent_uuid: "10d07cb0cc8840bea5ce1292f24169f3" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 34085 } }
I20260807 15:11:52.998487 8797 raft_consensus.cc:348] T 1c90d746d96b492fad569026a4cebe61 P 9d215d5c47fd4619ad846c7151ad85c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d215d5c47fd4619ad846c7151ad85c1" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 39783 } } peers { permanent_uuid: "1a18fd2e4123445cad3e1fa79a7fb96c" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 32991 } } peers { permanent_uuid: "10d07cb0cc8840bea5ce1292f24169f3" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 34085 } }
I20260807 15:11:52.998658 8797 raft_consensus.cc:374] T 1c90d746d96b492fad569026a4cebe61 P 9d215d5c47fd4619ad846c7151ad85c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:11:52.998703 8797 raft_consensus.cc:729] T 1c90d746d96b492fad569026a4cebe61 P 9d215d5c47fd4619ad846c7151ad85c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9d215d5c47fd4619ad846c7151ad85c1, State: Initialized, Role: FOLLOWER
I20260807 15:11:52.998718 8799 ts_tablet_manager.cc:1428] T 1c90d746d96b492fad569026a4cebe61 P 10d07cb0cc8840bea5ce1292f24169f3: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260807 15:11:52.998812 8797 consensus_queue.cc:260] T 1c90d746d96b492fad569026a4cebe61 P 9d215d5c47fd4619ad846c7151ad85c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d215d5c47fd4619ad846c7151ad85c1" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 39783 } } peers { permanent_uuid: "1a18fd2e4123445cad3e1fa79a7fb96c" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 32991 } } peers { permanent_uuid: "10d07cb0cc8840bea5ce1292f24169f3" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 34085 } }
I20260807 15:11:52.998880 8571 heartbeater.cc:503] Master 127.29.194.62:39853 was elected leader, sending a full tablet report...
I20260807 15:11:52.999060 8797 ts_tablet_manager.cc:1428] T 1c90d746d96b492fad569026a4cebe61 P 9d215d5c47fd4619ad846c7151ad85c1: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260807 15:11:52.999069 8375 heartbeater.cc:503] Master 127.29.194.62:39853 was elected leader, sending a full tablet report...
I20260807 15:11:52.999804 8757 heartbeater.cc:503] Master 127.29.194.62:39853 was elected leader, sending a full tablet report...
I20260807 15:11:53.001652 8803 ts_tablet_manager.cc:1428] T 1c90d746d96b492fad569026a4cebe61 P 1a18fd2e4123445cad3e1fa79a7fb96c: Time spent starting tablet: real 0.006s user 0.003s sys 0.000s
I20260807 15:11:53.050127 8809 raft_consensus.cc:482] T 1c90d746d96b492fad569026a4cebe61 P 10d07cb0cc8840bea5ce1292f24169f3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 15:11:53.050264 8809 raft_consensus.cc:504] T 1c90d746d96b492fad569026a4cebe61 P 10d07cb0cc8840bea5ce1292f24169f3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d215d5c47fd4619ad846c7151ad85c1" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 39783 } } peers { permanent_uuid: "1a18fd2e4123445cad3e1fa79a7fb96c" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 32991 } } peers { permanent_uuid: "10d07cb0cc8840bea5ce1292f24169f3" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 34085 } }
I20260807 15:11:53.050657 8809 leader_election.cc:288] T 1c90d746d96b492fad569026a4cebe61 P 10d07cb0cc8840bea5ce1292f24169f3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9d215d5c47fd4619ad846c7151ad85c1 (127.29.194.3:39783), 1a18fd2e4123445cad3e1fa79a7fb96c (127.29.194.2:32991)
I20260807 15:11:53.053419 8795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:53.050922 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.194.1:47543 (local address 127.29.194.3:39783)
0807 15:11:53.050977 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:53.050979 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:53.050990 (+ 11us) server_negotiation.cc:407] Connection header received
0807 15:11:53.051023 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:53.051024 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:53.051034 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:53.051086 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:11:53.051248 (+ 162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:53.051746 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:53.052708 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:53.053083 (+ 375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:53.053150 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:53.053161 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0807 15:11:53.053307 (+ 146us) server_negotiation.cc:299] Negotiation successful
0807 15:11:53.053343 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260807 15:11:53.053419 8380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:53.050850 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.3:39783 (local address 127.29.194.1:47543)
0807 15:11:53.050907 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0807 15:11:53.050918 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 15:11:53.050965 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:11:53.051102 (+ 137us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:11:53.051103 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:11:53.051104 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:11:53.051185 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:53.051187 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:53.051837 (+ 650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:53.051838 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:53.052607 (+ 769us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:53.052610 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:53.053165 (+ 555us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:53.053167 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:53.053241 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:53.053245 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 15:11:53.053290 (+ 45us) client_negotiation.cc:240] Negotiation successful
0807 15:11:53.053339 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38}
I20260807 15:11:53.053560 8792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:53.051080 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.194.1:47443 (local address 127.29.194.2:32991)
0807 15:11:53.051122 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:53.051124 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:53.051133 (+ 9us) server_negotiation.cc:407] Connection header received
0807 15:11:53.051161 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:53.051162 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:53.051170 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:53.051216 (+ 46us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:11:53.051371 (+ 155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:53.051893 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:53.052797 (+ 904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:53.053183 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:53.053244 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:53.053253 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0807 15:11:53.053421 (+ 168us) server_negotiation.cc:299] Negotiation successful
0807 15:11:53.053500 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260807 15:11:53.053556 8382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:53.050916 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.2:32991 (local address 127.29.194.1:47443)
0807 15:11:53.051021 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0807 15:11:53.051033 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 15:11:53.051087 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:11:53.051234 (+ 147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:11:53.051235 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:11:53.051236 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:11:53.051304 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:53.051306 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:53.051984 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:53.051985 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:53.052704 (+ 719us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:53.052706 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:53.053263 (+ 557us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:53.053264 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:53.053347 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:53.053350 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 15:11:53.053401 (+ 51us) client_negotiation.cc:240] Negotiation successful
0807 15:11:53.053472 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":82}
I20260807 15:11:53.053840 8694 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c90d746d96b492fad569026a4cebe61" candidate_uuid: "10d07cb0cc8840bea5ce1292f24169f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d215d5c47fd4619ad846c7151ad85c1" is_pre_election: true
I20260807 15:11:53.053912 8502 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c90d746d96b492fad569026a4cebe61" candidate_uuid: "10d07cb0cc8840bea5ce1292f24169f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1a18fd2e4123445cad3e1fa79a7fb96c" is_pre_election: true
I20260807 15:11:53.054013 8694 raft_consensus.cc:2454] T 1c90d746d96b492fad569026a4cebe61 P 9d215d5c47fd4619ad846c7151ad85c1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d07cb0cc8840bea5ce1292f24169f3 in term 0.
I20260807 15:11:53.054072 8502 raft_consensus.cc:2454] T 1c90d746d96b492fad569026a4cebe61 P 1a18fd2e4123445cad3e1fa79a7fb96c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d07cb0cc8840bea5ce1292f24169f3 in term 0.
I20260807 15:11:53.054373 8212 leader_election.cc:302] T 1c90d746d96b492fad569026a4cebe61 P 10d07cb0cc8840bea5ce1292f24169f3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10d07cb0cc8840bea5ce1292f24169f3, 9d215d5c47fd4619ad846c7151ad85c1; no voters:
I20260807 15:11:53.054548 8809 raft_consensus.cc:2781] T 1c90d746d96b492fad569026a4cebe61 P 10d07cb0cc8840bea5ce1292f24169f3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 15:11:53.054592 8809 raft_consensus.cc:482] T 1c90d746d96b492fad569026a4cebe61 P 10d07cb0cc8840bea5ce1292f24169f3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 15:11:53.054615 8809 raft_consensus.cc:3037] T 1c90d746d96b492fad569026a4cebe61 P 10d07cb0cc8840bea5ce1292f24169f3 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:11:53.055373 8809 raft_consensus.cc:504] T 1c90d746d96b492fad569026a4cebe61 P 10d07cb0cc8840bea5ce1292f24169f3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d215d5c47fd4619ad846c7151ad85c1" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 39783 } } peers { permanent_uuid: "1a18fd2e4123445cad3e1fa79a7fb96c" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 32991 } } peers { permanent_uuid: "10d07cb0cc8840bea5ce1292f24169f3" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 34085 } }
I20260807 15:11:53.055519 8809 leader_election.cc:288] T 1c90d746d96b492fad569026a4cebe61 P 10d07cb0cc8840bea5ce1292f24169f3 [CANDIDATE]: Term 1 election: Requested vote from peers 9d215d5c47fd4619ad846c7151ad85c1 (127.29.194.3:39783), 1a18fd2e4123445cad3e1fa79a7fb96c (127.29.194.2:32991)
I20260807 15:11:53.055728 8502 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c90d746d96b492fad569026a4cebe61" candidate_uuid: "10d07cb0cc8840bea5ce1292f24169f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1a18fd2e4123445cad3e1fa79a7fb96c"
I20260807 15:11:53.055781 8694 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c90d746d96b492fad569026a4cebe61" candidate_uuid: "10d07cb0cc8840bea5ce1292f24169f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d215d5c47fd4619ad846c7151ad85c1"
I20260807 15:11:53.055826 8502 raft_consensus.cc:3037] T 1c90d746d96b492fad569026a4cebe61 P 1a18fd2e4123445cad3e1fa79a7fb96c [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:11:53.055853 8694 raft_consensus.cc:3037] T 1c90d746d96b492fad569026a4cebe61 P 9d215d5c47fd4619ad846c7151ad85c1 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:11:53.056756 8694 raft_consensus.cc:2454] T 1c90d746d96b492fad569026a4cebe61 P 9d215d5c47fd4619ad846c7151ad85c1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d07cb0cc8840bea5ce1292f24169f3 in term 1.
I20260807 15:11:53.056756 8502 raft_consensus.cc:2454] T 1c90d746d96b492fad569026a4cebe61 P 1a18fd2e4123445cad3e1fa79a7fb96c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d07cb0cc8840bea5ce1292f24169f3 in term 1.
I20260807 15:11:53.057014 8212 leader_election.cc:302] T 1c90d746d96b492fad569026a4cebe61 P 10d07cb0cc8840bea5ce1292f24169f3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10d07cb0cc8840bea5ce1292f24169f3, 9d215d5c47fd4619ad846c7151ad85c1; no voters:
I20260807 15:11:53.057176 8809 raft_consensus.cc:2781] T 1c90d746d96b492fad569026a4cebe61 P 10d07cb0cc8840bea5ce1292f24169f3 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:11:53.057336 8809 raft_consensus.cc:686] T 1c90d746d96b492fad569026a4cebe61 P 10d07cb0cc8840bea5ce1292f24169f3 [term 1 LEADER]: Becoming Leader. State: Replica: 10d07cb0cc8840bea5ce1292f24169f3, State: Running, Role: LEADER
I20260807 15:11:53.057418 8809 consensus_queue.cc:237] T 1c90d746d96b492fad569026a4cebe61 P 10d07cb0cc8840bea5ce1292f24169f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d215d5c47fd4619ad846c7151ad85c1" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 39783 } } peers { permanent_uuid: "1a18fd2e4123445cad3e1fa79a7fb96c" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 32991 } } peers { permanent_uuid: "10d07cb0cc8840bea5ce1292f24169f3" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 34085 } }
I20260807 15:11:53.058624 7010 catalog_manager.cc:5495] T 1c90d746d96b492fad569026a4cebe61 P 10d07cb0cc8840bea5ce1292f24169f3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 10d07cb0cc8840bea5ce1292f24169f3 (127.29.194.1). New cstate: current_term: 1 leader_uuid: "10d07cb0cc8840bea5ce1292f24169f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d215d5c47fd4619ad846c7151ad85c1" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 39783 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1a18fd2e4123445cad3e1fa79a7fb96c" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 32991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "10d07cb0cc8840bea5ce1292f24169f3" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 34085 } health_report { overall_health: HEALTHY } } }
W20260807 15:11:53.059964 8573 tablet.cc:2369] T 1c90d746d96b492fad569026a4cebe61 P 1a18fd2e4123445cad3e1fa79a7fb96c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 15:11:53.082053 8796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:53.079387 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38682 (local address 127.29.194.1:34085)
0807 15:11:53.079438 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:53.079440 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:53.079448 (+ 8us) server_negotiation.cc:407] Connection header received
0807 15:11:53.079495 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:53.079496 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:53.079560 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:53.079638 (+ 78us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 15:11:53.079798 (+ 160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:53.080306 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:53.081126 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:53.081332 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:53.081417 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:53.081687 (+ 270us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 15:11:53.081812 (+ 125us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 15:11:53.081895 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0807 15:11:53.081966 (+ 71us) server_negotiation.cc:299] Negotiation successful
0807 15:11:53.081988 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260807 15:11:53.085300 8694 raft_consensus.cc:1261] T 1c90d746d96b492fad569026a4cebe61 P 9d215d5c47fd4619ad846c7151ad85c1 [term 1 FOLLOWER]: Refusing update from remote peer 10d07cb0cc8840bea5ce1292f24169f3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:11:53.085274 8502 raft_consensus.cc:1261] T 1c90d746d96b492fad569026a4cebe61 P 1a18fd2e4123445cad3e1fa79a7fb96c [term 1 FOLLOWER]: Refusing update from remote peer 10d07cb0cc8840bea5ce1292f24169f3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:11:53.085934 8814 consensus_queue.cc:1035] T 1c90d746d96b492fad569026a4cebe61 P 10d07cb0cc8840bea5ce1292f24169f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9d215d5c47fd4619ad846c7151ad85c1" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 39783 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:11:53.086119 8809 consensus_queue.cc:1035] T 1c90d746d96b492fad569026a4cebe61 P 10d07cb0cc8840bea5ce1292f24169f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1a18fd2e4123445cad3e1fa79a7fb96c" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 32991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:11:53.090615 8819 mvcc.cc:205] Tried to move back new op lower bound from 7315929141592174592 to 7315929141484236800. Current Snapshot: MvccSnapshot[applied={T|T < 7315929141592174592}]
I20260807 15:11:53.093473 7010 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43364:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 15:11:53.093590 7010 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43364:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 15:11:53.095278 7010 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4ebd9481251f4a29ba10f88819ba9ae2: Sending DeleteTablet for 3 replicas of tablet 1c90d746d96b492fad569026a4cebe61
I20260807 15:11:53.095534 8666 tablet_service.cc:1514] Processing DeleteTablet for tablet 1c90d746d96b492fad569026a4cebe61 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:11:53 UTC) from {username='oryx', principal='oryx/127.29.194.62@KRBTEST.COM'} at 127.0.0.1:47964
I20260807 15:11:53.095703 8473 tablet_service.cc:1514] Processing DeleteTablet for tablet 1c90d746d96b492fad569026a4cebe61 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:11:53 UTC) from {username='oryx', principal='oryx/127.29.194.62@KRBTEST.COM'} at 127.0.0.1:39864
I20260807 15:11:53.095862 8830 tablet_replica.cc:330] T 1c90d746d96b492fad569026a4cebe61 P 9d215d5c47fd4619ad846c7151ad85c1: stopping tablet replica
I20260807 15:11:53.096028 8830 raft_consensus.cc:2229] T 1c90d746d96b492fad569026a4cebe61 P 9d215d5c47fd4619ad846c7151ad85c1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 15:11:53.096006 8278 tablet_service.cc:1514] Processing DeleteTablet for tablet 1c90d746d96b492fad569026a4cebe61 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:11:53 UTC) from {username='oryx', principal='oryx/127.29.194.62@KRBTEST.COM'} at 127.0.0.1:38680
I20260807 15:11:53.096071 8666 tablet_service.cc:1514] Processing DeleteTablet for tablet 1c90d746d96b492fad569026a4cebe61 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:11:53 UTC) from {username='oryx', principal='oryx/127.29.194.62@KRBTEST.COM'} at 127.0.0.1:47964
I20260807 15:11:53.096365 8831 tablet_replica.cc:330] T 1c90d746d96b492fad569026a4cebe61 P 1a18fd2e4123445cad3e1fa79a7fb96c: stopping tablet replica
I20260807 15:11:53.096447 8831 raft_consensus.cc:2229] T 1c90d746d96b492fad569026a4cebe61 P 1a18fd2e4123445cad3e1fa79a7fb96c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 15:11:53.096525 8831 raft_consensus.cc:2258] T 1c90d746d96b492fad569026a4cebe61 P 1a18fd2e4123445cad3e1fa79a7fb96c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:11:53.097352 30472 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7fZSIL/build/release/bin/kudu with pid 6965
I20260807 15:11:53.097340 8830 raft_consensus.cc:2258] T 1c90d746d96b492fad569026a4cebe61 P 9d215d5c47fd4619ad846c7151ad85c1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:11:53.105300 8831 ts_tablet_manager.cc:1905] T 1c90d746d96b492fad569026a4cebe61 P 1a18fd2e4123445cad3e1fa79a7fb96c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:11:53.106269 8830 ts_tablet_manager.cc:1905] T 1c90d746d96b492fad569026a4cebe61 P 9d215d5c47fd4619ad846c7151ad85c1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:11:53.108083 8830 ts_tablet_manager.cc:1918] T 1c90d746d96b492fad569026a4cebe61 P 9d215d5c47fd4619ad846c7151ad85c1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 15:11:53.108175 8830 log.cc:1196] T 1c90d746d96b492fad569026a4cebe61 P 9d215d5c47fd4619ad846c7151ad85c1: Deleting WAL directory at /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115480851507-30472-0/minicluster-data/ts-2/wal/wals/1c90d746d96b492fad569026a4cebe61
I20260807 15:11:53.108453 8830 ts_tablet_manager.cc:1939] T 1c90d746d96b492fad569026a4cebe61 P 9d215d5c47fd4619ad846c7151ad85c1: Deleting consensus metadata
I20260807 15:11:53.109366 8832 tablet_replica.cc:330] T 1c90d746d96b492fad569026a4cebe61 P 10d07cb0cc8840bea5ce1292f24169f3: stopping tablet replica
I20260807 15:11:53.109597 8832 raft_consensus.cc:2229] T 1c90d746d96b492fad569026a4cebe61 P 10d07cb0cc8840bea5ce1292f24169f3 [term 1 LEADER]: Raft consensus shutting down.
I20260807 15:11:53.109771 8832 raft_consensus.cc:2258] T 1c90d746d96b492fad569026a4cebe61 P 10d07cb0cc8840bea5ce1292f24169f3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:11:53.110711 8832 ts_tablet_manager.cc:1905] T 1c90d746d96b492fad569026a4cebe61 P 10d07cb0cc8840bea5ce1292f24169f3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:11:53.113415 8832 ts_tablet_manager.cc:1918] T 1c90d746d96b492fad569026a4cebe61 P 10d07cb0cc8840bea5ce1292f24169f3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 15:11:53.113579 8832 log.cc:1196] T 1c90d746d96b492fad569026a4cebe61 P 10d07cb0cc8840bea5ce1292f24169f3: Deleting WAL directory at /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115480851507-30472-0/minicluster-data/ts-0/wal/wals/1c90d746d96b492fad569026a4cebe61
I20260807 15:11:53.113855 8832 ts_tablet_manager.cc:1939] T 1c90d746d96b492fad569026a4cebe61 P 10d07cb0cc8840bea5ce1292f24169f3: Deleting consensus metadata
I20260807 15:11:53.115216 8831 ts_tablet_manager.cc:1918] T 1c90d746d96b492fad569026a4cebe61 P 1a18fd2e4123445cad3e1fa79a7fb96c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 15:11:53.115288 8381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:53.097179 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33826 (local address 127.29.194.60:44209)
0807 15:11:53.097252 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:53.097254 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:53.105770 (+ 8516us) server_negotiation.cc:407] Connection header received
0807 15:11:53.106146 (+ 376us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:53.106149 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:53.106168 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:53.106204 (+ 36us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:11:53.107198 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:53.108102 (+ 904us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:53.109274 (+ 1172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:53.109670 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:53.109779 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:53.112232 (+ 2453us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:11:53.112234 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:11:53.112234 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:11:53.112242 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:11:53.113930 (+ 1688us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:53.114356 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:53.114357 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:53.114358 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:53.114418 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:53.114625 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:53.114627 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:53.114627 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:53.114792 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:11:53.114924 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0807 15:11:53.115068 (+ 144us) server_negotiation.cc:299] Negotiation successful
0807 15:11:53.115196 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260807 15:11:53.115309 8831 log.cc:1196] T 1c90d746d96b492fad569026a4cebe61 P 1a18fd2e4123445cad3e1fa79a7fb96c: Deleting WAL directory at /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115480851507-30472-0/minicluster-data/ts-1/wal/wals/1c90d746d96b492fad569026a4cebe61
I20260807 15:11:53.115573 8831 ts_tablet_manager.cc:1939] T 1c90d746d96b492fad569026a4cebe61 P 1a18fd2e4123445cad3e1fa79a7fb96c: Deleting consensus metadata
W20260807 15:11:53.116019 8212 connection.cc:535] server connection from 127.0.0.1:38680 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260807 15:11:53.116240 8414 connection.cc:535] server connection from 127.0.0.1:39864 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260807 15:11:53.116312 8598 connection.cc:535] server connection from 127.0.0.1:47964 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260807 15:11:53.127540 8777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:53.119251 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51726 (local address 127.29.194.61:40955)
0807 15:11:53.119377 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:53.119380 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:53.119678 (+ 298us) server_negotiation.cc:407] Connection header received
0807 15:11:53.120043 (+ 365us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:53.120045 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:53.120064 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:53.120121 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:11:53.121104 (+ 983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:53.121893 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:53.122843 (+ 950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:53.123203 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:53.123272 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:53.124647 (+ 1375us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:11:53.124649 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:11:53.124649 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:11:53.124657 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:11:53.126307 (+ 1650us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:53.126785 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:53.126786 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:53.126786 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:53.126847 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:53.127025 (+ 178us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:53.127026 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:53.127027 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:53.127149 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:11:53.127234 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0807 15:11:53.127336 (+ 102us) server_negotiation.cc:299] Negotiation successful
0807 15:11:53.127431 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94}
I20260807 15:11:53.127671 8381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:53.119177 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33830 (local address 127.29.194.60:44209)
0807 15:11:53.119296 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:53.119298 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:53.119782 (+ 484us) server_negotiation.cc:407] Connection header received
0807 15:11:53.119828 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:53.119830 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:53.119845 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:53.119943 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:11:53.120795 (+ 852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:53.121717 (+ 922us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:53.122873 (+ 1156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:53.123200 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:53.123271 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:53.124896 (+ 1625us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:11:53.124897 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:11:53.124897 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:11:53.124905 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:11:53.126430 (+ 1525us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:53.126814 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:53.126815 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:53.126815 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:53.126879 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:53.127090 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:53.127091 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:53.127091 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:53.127229 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:11:53.127297 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0807 15:11:53.127490 (+ 193us) server_negotiation.cc:299] Negotiation successful
0807 15:11:53.127575 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260807 15:11:54.094172 8848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:54.093846 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:39853 (local address 127.29.194.2:33397)
0807 15:11:54.093983 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0807 15:11:54.094070 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.194.62:39853: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":76,"threads_started":1}
W20260807 15:11:54.094619 8571 heartbeater.cc:644] Failed to heartbeat to 127.29.194.62:39853 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.29.194.62:39853: connect: Connection refused (error 111)
I20260807 15:11:54.094731 8850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:54.094339 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:39853 (local address 127.29.194.1:58869)
0807 15:11:54.094483 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0807 15:11:54.094629 (+ 146us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.194.62:39853: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":56,"threads_started":1}
W20260807 15:11:54.094892 8375 heartbeater.cc:644] Failed to heartbeat to 127.29.194.62:39853 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.29.194.62:39853: connect: Connection refused (error 111)
I20260807 15:11:54.096866 8852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:54.096612 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:39853 (local address 127.29.194.3:43319)
0807 15:11:54.096753 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0807 15:11:54.096825 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.194.62:39853: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":60,"threads_started":1}
W20260807 15:11:54.097119 8757 heartbeater.cc:644] Failed to heartbeat to 127.29.194.62:39853 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.29.194.62:39853: connect: Connection refused (error 111)
I20260807 15:11:54.595355 8855 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8e78c2a81ab3476b8cbeff1833835aeb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4ebd9481251f4a29ba10f88819ba9ae2)
I20260807 15:11:54.595378 8853 raft_consensus.cc:482] T 00000000000000000000000000000000 P e5603c6f9ea84f1a9510499da5950324 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4ebd9481251f4a29ba10f88819ba9ae2)
I20260807 15:11:54.595526 8855 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8e78c2a81ab3476b8cbeff1833835aeb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "4ebd9481251f4a29ba10f88819ba9ae2" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 39853 } } peers { permanent_uuid: "8e78c2a81ab3476b8cbeff1833835aeb" member_type: VOTER last_known_addr { host: "127.29.194.61" port: 40955 } attrs { promote: false } } peers { permanent_uuid: "e5603c6f9ea84f1a9510499da5950324" member_type: VOTER last_known_addr { host: "127.29.194.60" port: 44209 } attrs { promote: false } }
I20260807 15:11:54.595700 8853 raft_consensus.cc:504] T 00000000000000000000000000000000 P e5603c6f9ea84f1a9510499da5950324 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "4ebd9481251f4a29ba10f88819ba9ae2" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 39853 } } peers { permanent_uuid: "8e78c2a81ab3476b8cbeff1833835aeb" member_type: VOTER last_known_addr { host: "127.29.194.61" port: 40955 } attrs { promote: false } } peers { permanent_uuid: "e5603c6f9ea84f1a9510499da5950324" member_type: VOTER last_known_addr { host: "127.29.194.60" port: 44209 } attrs { promote: false } }
I20260807 15:11:54.596268 8855 leader_election.cc:288] T 00000000000000000000000000000000 P 8e78c2a81ab3476b8cbeff1833835aeb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4ebd9481251f4a29ba10f88819ba9ae2 (127.29.194.62:39853), e5603c6f9ea84f1a9510499da5950324 (127.29.194.60:44209)
I20260807 15:11:54.596333 8859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:54.596007 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:39853 (local address 127.0.0.1:43828)
0807 15:11:54.596116 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0807 15:11:54.596295 (+ 179us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.194.62:39853: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":90,"thread_start_us":46,"threads_started":1}
I20260807 15:11:54.596405 8853 leader_election.cc:288] T 00000000000000000000000000000000 P e5603c6f9ea84f1a9510499da5950324 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4ebd9481251f4a29ba10f88819ba9ae2 (127.29.194.62:39853), 8e78c2a81ab3476b8cbeff1833835aeb (127.29.194.61:40955)
W20260807 15:11:54.596426 7839 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.29.194.62:39853: connect: Connection refused (error 111)
I20260807 15:11:54.596628 8862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:54.596177 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:39853 (local address 127.0.0.1:43838)
0807 15:11:54.596536 (+ 359us) negotiation.cc:107] Waiting for socket to connect
0807 15:11:54.596593 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.194.62:39853: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":346,"thread_start_us":62,"threads_started":1}
I20260807 15:11:54.596827 8859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:54.596750 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:39853 (local address 127.0.0.1:43852)
0807 15:11:54.596782 (+ 32us) negotiation.cc:107] Waiting for socket to connect
0807 15:11:54.596803 (+ 21us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.194.62:39853: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":21}
W20260807 15:11:54.596819 8105 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.29.194.62:39853: connect: Connection refused (error 111)
W20260807 15:11:54.596920 7839 leader_election.cc:334] T 00000000000000000000000000000000 P 8e78c2a81ab3476b8cbeff1833835aeb [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 4ebd9481251f4a29ba10f88819ba9ae2 (127.29.194.62:39853): Network error: Client connection negotiation failed: client connection to 127.29.194.62:39853: connect: Connection refused (error 111)
I20260807 15:11:54.597566 8866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:54.597350 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:39853 (local address 127.0.0.1:43860)
0807 15:11:54.597495 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0807 15:11:54.597533 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.194.62:39853: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":73,"threads_started":1}
W20260807 15:11:54.597648 8105 leader_election.cc:334] T 00000000000000000000000000000000 P e5603c6f9ea84f1a9510499da5950324 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 4ebd9481251f4a29ba10f88819ba9ae2 (127.29.194.62:39853): Network error: Client connection negotiation failed: client connection to 127.29.194.62:39853: connect: Connection refused (error 111)
I20260807 15:11:54.599213 8857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:54.596042 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.60:44209 (local address 127.0.0.1:33832)
0807 15:11:54.596193 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0807 15:11:54.596205 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 15:11:54.596261 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:11:54.596675 (+ 414us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:11:54.596676 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:11:54.596678 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:11:54.596736 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:54.596739 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:54.597558 (+ 819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:54.597559 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:54.598443 (+ 884us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:54.598446 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:54.598996 (+ 550us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:54.598997 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:54.599051 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:54.599054 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 15:11:54.599097 (+ 43us) client_negotiation.cc:240] Negotiation successful
0807 15:11:54.599135 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":38,"threads_started":1}
I20260807 15:11:54.599282 8863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:54.596332 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51730 (local address 127.29.194.61:40955)
0807 15:11:54.596502 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:54.596504 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:54.596513 (+ 9us) server_negotiation.cc:407] Connection header received
0807 15:11:54.596537 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:54.596538 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:54.596549 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:54.596621 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:11:54.596819 (+ 198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:54.597681 (+ 862us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:54.598607 (+ 926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:54.598956 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:54.599032 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:54.599042 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0807 15:11:54.599150 (+ 108us) server_negotiation.cc:299] Negotiation successful
0807 15:11:54.599183 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":155,"thread_start_us":49,"threads_started":1}
I20260807 15:11:54.599226 8861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:54.596160 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33832 (local address 127.29.194.60:44209)
0807 15:11:54.596551 (+ 391us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:54.596553 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:54.596560 (+ 7us) server_negotiation.cc:407] Connection header received
0807 15:11:54.596581 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:54.596581 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:54.596589 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:54.596799 (+ 210us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:11:54.596833 (+ 34us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:54.597417 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:54.598540 (+ 1123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:54.598923 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:54.598977 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:54.598986 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0807 15:11:54.599112 (+ 126us) server_negotiation.cc:299] Negotiation successful
0807 15:11:54.599159 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":376,"thread_start_us":79,"threads_started":1}
I20260807 15:11:54.599233 8860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:54.596171 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.61:40955 (local address 127.0.0.1:51730)
0807 15:11:54.596408 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0807 15:11:54.596419 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 15:11:54.596478 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:11:54.596687 (+ 209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:11:54.596688 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:11:54.596690 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:11:54.596745 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:54.596748 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:54.597762 (+ 1014us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:54.597763 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:54.598524 (+ 761us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:54.598527 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:54.599041 (+ 514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:54.599042 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:54.599086 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:54.599089 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 15:11:54.599138 (+ 49us) client_negotiation.cc:240] Negotiation successful
0807 15:11:54.599176 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":223,"thread_start_us":77,"threads_started":1}
I20260807 15:11:54.599565 8138 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "8e78c2a81ab3476b8cbeff1833835aeb" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "e5603c6f9ea84f1a9510499da5950324" is_pre_election: true
I20260807 15:11:54.599593 7868 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e5603c6f9ea84f1a9510499da5950324" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "8e78c2a81ab3476b8cbeff1833835aeb" is_pre_election: true
I20260807 15:11:54.599691 8138 raft_consensus.cc:2454] T 00000000000000000000000000000000 P e5603c6f9ea84f1a9510499da5950324 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e78c2a81ab3476b8cbeff1833835aeb in term 1.
I20260807 15:11:54.599691 7868 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 8e78c2a81ab3476b8cbeff1833835aeb [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5603c6f9ea84f1a9510499da5950324 in term 1.
I20260807 15:11:54.599941 8103 leader_election.cc:302] T 00000000000000000000000000000000 P e5603c6f9ea84f1a9510499da5950324 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8e78c2a81ab3476b8cbeff1833835aeb, e5603c6f9ea84f1a9510499da5950324; no voters: 4ebd9481251f4a29ba10f88819ba9ae2
I20260807 15:11:54.599941 7835 leader_election.cc:302] T 00000000000000000000000000000000 P 8e78c2a81ab3476b8cbeff1833835aeb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8e78c2a81ab3476b8cbeff1833835aeb, e5603c6f9ea84f1a9510499da5950324; no voters: 4ebd9481251f4a29ba10f88819ba9ae2
I20260807 15:11:54.600171 8853 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e5603c6f9ea84f1a9510499da5950324 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260807 15:11:54.600222 8853 raft_consensus.cc:482] T 00000000000000000000000000000000 P e5603c6f9ea84f1a9510499da5950324 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 4ebd9481251f4a29ba10f88819ba9ae2)
I20260807 15:11:54.600241 8853 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e5603c6f9ea84f1a9510499da5950324 [term 1 FOLLOWER]: Advancing to term 2
I20260807 15:11:54.600238 8855 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8e78c2a81ab3476b8cbeff1833835aeb [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260807 15:11:54.600298 8855 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8e78c2a81ab3476b8cbeff1833835aeb [term 1 FOLLOWER]: Starting leader election (detected failure of leader 4ebd9481251f4a29ba10f88819ba9ae2)
I20260807 15:11:54.600325 8855 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8e78c2a81ab3476b8cbeff1833835aeb [term 1 FOLLOWER]: Advancing to term 2
I20260807 15:11:54.601128 8855 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8e78c2a81ab3476b8cbeff1833835aeb [term 2 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "4ebd9481251f4a29ba10f88819ba9ae2" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 39853 } } peers { permanent_uuid: "8e78c2a81ab3476b8cbeff1833835aeb" member_type: VOTER last_known_addr { host: "127.29.194.61" port: 40955 } attrs { promote: false } } peers { permanent_uuid: "e5603c6f9ea84f1a9510499da5950324" member_type: VOTER last_known_addr { host: "127.29.194.60" port: 44209 } attrs { promote: false } }
I20260807 15:11:54.601226 8853 raft_consensus.cc:504] T 00000000000000000000000000000000 P e5603c6f9ea84f1a9510499da5950324 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "4ebd9481251f4a29ba10f88819ba9ae2" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 39853 } } peers { permanent_uuid: "8e78c2a81ab3476b8cbeff1833835aeb" member_type: VOTER last_known_addr { host: "127.29.194.61" port: 40955 } attrs { promote: false } } peers { permanent_uuid: "e5603c6f9ea84f1a9510499da5950324" member_type: VOTER last_known_addr { host: "127.29.194.60" port: 44209 } attrs { promote: false } }
I20260807 15:11:54.601284 8855 leader_election.cc:288] T 00000000000000000000000000000000 P 8e78c2a81ab3476b8cbeff1833835aeb [CANDIDATE]: Term 2 election: Requested vote from peers 4ebd9481251f4a29ba10f88819ba9ae2 (127.29.194.62:39853), e5603c6f9ea84f1a9510499da5950324 (127.29.194.60:44209)
I20260807 15:11:54.601524 8138 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "8e78c2a81ab3476b8cbeff1833835aeb" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "e5603c6f9ea84f1a9510499da5950324"
I20260807 15:11:54.601572 8857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:54.601463 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:39853 (local address 127.0.0.1:43876)
0807 15:11:54.601517 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0807 15:11:54.601546 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.194.62:39853: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
I20260807 15:11:54.601635 8138 raft_consensus.cc:2379] T 00000000000000000000000000000000 P e5603c6f9ea84f1a9510499da5950324 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 8e78c2a81ab3476b8cbeff1833835aeb in current term 2: Already voted for candidate e5603c6f9ea84f1a9510499da5950324 in this term.
I20260807 15:11:54.601776 8860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:54.601592 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:39853 (local address 127.0.0.1:43884)
0807 15:11:54.601708 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0807 15:11:54.601749 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.194.62:39853: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":86}
I20260807 15:11:54.601891 8857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:54.601805 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:39853 (local address 127.0.0.1:43892)
0807 15:11:54.601848 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0807 15:11:54.601872 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.194.62:39853: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
I20260807 15:11:54.601886 7868 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e5603c6f9ea84f1a9510499da5950324" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "8e78c2a81ab3476b8cbeff1833835aeb"
I20260807 15:11:54.602015 7868 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 8e78c2a81ab3476b8cbeff1833835aeb [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate e5603c6f9ea84f1a9510499da5950324 in current term 2: Already voted for candidate 8e78c2a81ab3476b8cbeff1833835aeb in this term.
I20260807 15:11:54.602106 8853 leader_election.cc:288] T 00000000000000000000000000000000 P e5603c6f9ea84f1a9510499da5950324 [CANDIDATE]: Term 2 election: Requested vote from peers 4ebd9481251f4a29ba10f88819ba9ae2 (127.29.194.62:39853), 8e78c2a81ab3476b8cbeff1833835aeb (127.29.194.61:40955)
W20260807 15:11:54.602240 7839 leader_election.cc:334] T 00000000000000000000000000000000 P 8e78c2a81ab3476b8cbeff1833835aeb [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 4ebd9481251f4a29ba10f88819ba9ae2 (127.29.194.62:39853): Network error: Client connection negotiation failed: client connection to 127.29.194.62:39853: connect: Connection refused (error 111)
I20260807 15:11:54.602290 7839 leader_election.cc:302] T 00000000000000000000000000000000 P 8e78c2a81ab3476b8cbeff1833835aeb [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8e78c2a81ab3476b8cbeff1833835aeb; no voters: 4ebd9481251f4a29ba10f88819ba9ae2, e5603c6f9ea84f1a9510499da5950324
I20260807 15:11:54.602383 8855 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 8e78c2a81ab3476b8cbeff1833835aeb [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260807 15:11:54.602425 8860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:54.602177 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:39853 (local address 127.0.0.1:43900)
0807 15:11:54.602326 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0807 15:11:54.602361 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.194.62:39853: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126}
W20260807 15:11:54.602512 8105 leader_election.cc:334] T 00000000000000000000000000000000 P e5603c6f9ea84f1a9510499da5950324 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 4ebd9481251f4a29ba10f88819ba9ae2 (127.29.194.62:39853): Network error: Client connection negotiation failed: client connection to 127.29.194.62:39853: connect: Connection refused (error 111)
I20260807 15:11:54.602558 8105 leader_election.cc:302] T 00000000000000000000000000000000 P e5603c6f9ea84f1a9510499da5950324 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5603c6f9ea84f1a9510499da5950324; no voters: 4ebd9481251f4a29ba10f88819ba9ae2, 8e78c2a81ab3476b8cbeff1833835aeb
I20260807 15:11:54.602639 8853 raft_consensus.cc:2726] T 00000000000000000000000000000000 P e5603c6f9ea84f1a9510499da5950324 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260807 15:11:55.095643 8867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:55.095367 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:39853 (local address 127.29.194.2:35683)
0807 15:11:55.095545 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0807 15:11:55.095586 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.194.62:39853: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":76,"threads_started":1}
I20260807 15:11:55.095641 8868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:55.095388 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:39853 (local address 127.29.194.1:33483)
0807 15:11:55.095539 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0807 15:11:55.095588 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.194.62:39853: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":88,"threads_started":1}
I20260807 15:11:55.097743 8871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:55.097539 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:39853 (local address 127.29.194.3:35741)
0807 15:11:55.097649 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0807 15:11:55.097699 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.194.62:39853: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94,"thread_start_us":51,"threads_started":1}
I20260807 15:11:56.096366 8875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:56.096124 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:39853 (local address 127.29.194.1:60285)
0807 15:11:56.096272 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0807 15:11:56.096315 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.194.62:39853: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":70,"threads_started":1}
I20260807 15:11:56.096401 8878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:56.096124 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:39853 (local address 127.29.194.2:54851)
0807 15:11:56.096321 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0807 15:11:56.096361 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.194.62:39853: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":68,"threads_started":1}
I20260807 15:11:56.098588 8879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:56.098360 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:39853 (local address 127.29.194.3:51891)
0807 15:11:56.098509 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0807 15:11:56.098544 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.194.62:39853: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":73,"threads_started":1}
I20260807 15:11:56.117524 8880 raft_consensus.cc:482] T 00000000000000000000000000000000 P e5603c6f9ea84f1a9510499da5950324 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 15:11:56.117679 8880 raft_consensus.cc:504] T 00000000000000000000000000000000 P e5603c6f9ea84f1a9510499da5950324 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "4ebd9481251f4a29ba10f88819ba9ae2" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 39853 } } peers { permanent_uuid: "8e78c2a81ab3476b8cbeff1833835aeb" member_type: VOTER last_known_addr { host: "127.29.194.61" port: 40955 } attrs { promote: false } } peers { permanent_uuid: "e5603c6f9ea84f1a9510499da5950324" member_type: VOTER last_known_addr { host: "127.29.194.60" port: 44209 } attrs { promote: false } }
I20260807 15:11:56.117848 8880 leader_election.cc:288] T 00000000000000000000000000000000 P e5603c6f9ea84f1a9510499da5950324 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4ebd9481251f4a29ba10f88819ba9ae2 (127.29.194.62:39853), 8e78c2a81ab3476b8cbeff1833835aeb (127.29.194.61:40955)
I20260807 15:11:56.118160 7868 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e5603c6f9ea84f1a9510499da5950324" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "8e78c2a81ab3476b8cbeff1833835aeb" is_pre_election: true
I20260807 15:11:56.118330 7868 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 8e78c2a81ab3476b8cbeff1833835aeb [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5603c6f9ea84f1a9510499da5950324 in term 2.
I20260807 15:11:56.118368 8881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:56.118019 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:39853 (local address 127.0.0.1:44112)
0807 15:11:56.118169 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0807 15:11:56.118215 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.194.62:39853: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":84,"threads_started":1}
I20260807 15:11:56.118618 8103 leader_election.cc:302] T 00000000000000000000000000000000 P e5603c6f9ea84f1a9510499da5950324 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8e78c2a81ab3476b8cbeff1833835aeb, e5603c6f9ea84f1a9510499da5950324; no voters:
I20260807 15:11:56.118726 8880 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e5603c6f9ea84f1a9510499da5950324 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260807 15:11:56.118834 8880 raft_consensus.cc:482] T 00000000000000000000000000000000 P e5603c6f9ea84f1a9510499da5950324 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 15:11:56.118862 8880 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e5603c6f9ea84f1a9510499da5950324 [term 2 FOLLOWER]: Advancing to term 3
I20260807 15:11:56.119077 8881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:56.118959 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:39853 (local address 127.0.0.1:44120)
0807 15:11:56.119008 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0807 15:11:56.119051 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.194.62:39853: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
W20260807 15:11:56.119158 8105 leader_election.cc:334] T 00000000000000000000000000000000 P e5603c6f9ea84f1a9510499da5950324 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 4ebd9481251f4a29ba10f88819ba9ae2 (127.29.194.62:39853): Network error: Client connection negotiation failed: client connection to 127.29.194.62:39853: connect: Connection refused (error 111)
I20260807 15:11:56.119881 8880 raft_consensus.cc:504] T 00000000000000000000000000000000 P e5603c6f9ea84f1a9510499da5950324 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "4ebd9481251f4a29ba10f88819ba9ae2" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 39853 } } peers { permanent_uuid: "8e78c2a81ab3476b8cbeff1833835aeb" member_type: VOTER last_known_addr { host: "127.29.194.61" port: 40955 } attrs { promote: false } } peers { permanent_uuid: "e5603c6f9ea84f1a9510499da5950324" member_type: VOTER last_known_addr { host: "127.29.194.60" port: 44209 } attrs { promote: false } }
I20260807 15:11:56.120055 8880 leader_election.cc:288] T 00000000000000000000000000000000 P e5603c6f9ea84f1a9510499da5950324 [CANDIDATE]: Term 3 election: Requested vote from peers 4ebd9481251f4a29ba10f88819ba9ae2 (127.29.194.62:39853), 8e78c2a81ab3476b8cbeff1833835aeb (127.29.194.61:40955)
I20260807 15:11:56.120265 7868 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e5603c6f9ea84f1a9510499da5950324" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "8e78c2a81ab3476b8cbeff1833835aeb"
I20260807 15:11:56.120344 7868 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8e78c2a81ab3476b8cbeff1833835aeb [term 2 FOLLOWER]: Advancing to term 3
I20260807 15:11:56.120373 8881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:56.120256 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:39853 (local address 127.0.0.1:44126)
0807 15:11:56.120308 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0807 15:11:56.120343 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.194.62:39853: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
I20260807 15:11:56.121122 8881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:56.120961 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:39853 (local address 127.0.0.1:44138)
0807 15:11:56.121060 (+ 99us) negotiation.cc:107] Waiting for socket to connect
0807 15:11:56.121093 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.194.62:39853: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
W20260807 15:11:56.121236 8105 leader_election.cc:334] T 00000000000000000000000000000000 P e5603c6f9ea84f1a9510499da5950324 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 4ebd9481251f4a29ba10f88819ba9ae2 (127.29.194.62:39853): Network error: Client connection negotiation failed: client connection to 127.29.194.62:39853: connect: Connection refused (error 111)
I20260807 15:11:56.121325 7868 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 8e78c2a81ab3476b8cbeff1833835aeb [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5603c6f9ea84f1a9510499da5950324 in term 3.
I20260807 15:11:56.121665 8103 leader_election.cc:302] T 00000000000000000000000000000000 P e5603c6f9ea84f1a9510499da5950324 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8e78c2a81ab3476b8cbeff1833835aeb, e5603c6f9ea84f1a9510499da5950324; no voters: 4ebd9481251f4a29ba10f88819ba9ae2
I20260807 15:11:56.121809 8880 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e5603c6f9ea84f1a9510499da5950324 [term 3 FOLLOWER]: Leader election won for term 3
I20260807 15:11:56.122076 8880 raft_consensus.cc:686] T 00000000000000000000000000000000 P e5603c6f9ea84f1a9510499da5950324 [term 3 LEADER]: Becoming Leader. State: Replica: e5603c6f9ea84f1a9510499da5950324, State: Running, Role: LEADER
I20260807 15:11:56.122254 8880 consensus_queue.cc:237] T 00000000000000000000000000000000 P e5603c6f9ea84f1a9510499da5950324 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "4ebd9481251f4a29ba10f88819ba9ae2" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 39853 } } peers { permanent_uuid: "8e78c2a81ab3476b8cbeff1833835aeb" member_type: VOTER last_known_addr { host: "127.29.194.61" port: 40955 } attrs { promote: false } } peers { permanent_uuid: "e5603c6f9ea84f1a9510499da5950324" member_type: VOTER last_known_addr { host: "127.29.194.60" port: 44209 } attrs { promote: false } }
I20260807 15:11:56.122870 8883 sys_catalog.cc:455] T 00000000000000000000000000000000 P e5603c6f9ea84f1a9510499da5950324 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e5603c6f9ea84f1a9510499da5950324. Latest consensus state: current_term: 3 leader_uuid: "e5603c6f9ea84f1a9510499da5950324" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "4ebd9481251f4a29ba10f88819ba9ae2" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 39853 } } peers { permanent_uuid: "8e78c2a81ab3476b8cbeff1833835aeb" member_type: VOTER last_known_addr { host: "127.29.194.61" port: 40955 } attrs { promote: false } } peers { permanent_uuid: "e5603c6f9ea84f1a9510499da5950324" member_type: VOTER last_known_addr { host: "127.29.194.60" port: 44209 } attrs { promote: false } } }
I20260807 15:11:56.123008 8883 sys_catalog.cc:458] T 00000000000000000000000000000000 P e5603c6f9ea84f1a9510499da5950324 [sys.catalog]: This master's current role is: LEADER
I20260807 15:11:56.123284 8885 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:11:56.123987 8885 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:11:56.124073 8885 catalog_manager.cc:1259] Loaded cluster ID: b1e8937b172c41a9914d5edb2b5e84c5
I20260807 15:11:56.124089 8885 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:11:56.124322 8885 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:11:56.124406 8885 catalog_manager.cc:5879] T 00000000000000000000000000000000 P e5603c6f9ea84f1a9510499da5950324: Loaded TSK: 0
I20260807 15:11:56.124538 8885 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 15:11:56.156947 8124 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33830:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 15:11:56.159085 8124 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 15:11:56.160490 7868 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 8e78c2a81ab3476b8cbeff1833835aeb [term 3 FOLLOWER]: Refusing update from remote peer e5603c6f9ea84f1a9510499da5950324: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 3 index: 14. (index mismatch)
I20260807 15:11:56.160652 8881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:56.160448 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:39853 (local address 127.0.0.1:44142)
0807 15:11:56.160538 (+ 90us) negotiation.cc:107] Waiting for socket to connect
0807 15:11:56.160585 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.194.62:39853: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260807 15:11:56.161000 8883 consensus_queue.cc:1035] T 00000000000000000000000000000000 P e5603c6f9ea84f1a9510499da5950324 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e78c2a81ab3476b8cbeff1833835aeb" member_type: VOTER last_known_addr { host: "127.29.194.61" port: 40955 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260807 15:11:56.161320 8881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:56.161154 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:39853 (local address 127.0.0.1:44152)
0807 15:11:56.161231 (+ 77us) negotiation.cc:107] Waiting for socket to connect
0807 15:11:56.161281 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.194.62:39853: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260807 15:11:56.161434 8105 consensus_peers.cc:487] T 00000000000000000000000000000000 P e5603c6f9ea84f1a9510499da5950324 -> Peer 4ebd9481251f4a29ba10f88819ba9ae2 (127.29.194.62:39853): Couldn't send request to peer 4ebd9481251f4a29ba10f88819ba9ae2. Status: Network error: Client connection negotiation failed: client connection to 127.29.194.62:39853: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260807 15:11:56.162096 8888 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8e78c2a81ab3476b8cbeff1833835aeb [sys.catalog]: SysCatalogTable state changed. Reason: New leader e5603c6f9ea84f1a9510499da5950324. Latest consensus state: current_term: 3 leader_uuid: "e5603c6f9ea84f1a9510499da5950324" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "4ebd9481251f4a29ba10f88819ba9ae2" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 39853 } } peers { permanent_uuid: "8e78c2a81ab3476b8cbeff1833835aeb" member_type: VOTER last_known_addr { host: "127.29.194.61" port: 40955 } attrs { promote: false } } peers { permanent_uuid: "e5603c6f9ea84f1a9510499da5950324" member_type: VOTER last_known_addr { host: "127.29.194.60" port: 44209 } attrs { promote: false } } }
I20260807 15:11:56.162250 8888 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8e78c2a81ab3476b8cbeff1833835aeb [sys.catalog]: This master's current role is: FOLLOWER
I20260807 15:11:56.162995 8886 mvcc.cc:205] Tried to move back new op lower bound from 7315929154190274560 to 7315929154037682176. Current Snapshot: MvccSnapshot[applied={T|T < 7315929154190274560}]
I20260807 15:11:56.163112 8880 sys_catalog.cc:455] T 00000000000000000000000000000000 P e5603c6f9ea84f1a9510499da5950324 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "e5603c6f9ea84f1a9510499da5950324" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "4ebd9481251f4a29ba10f88819ba9ae2" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 39853 } } peers { permanent_uuid: "8e78c2a81ab3476b8cbeff1833835aeb" member_type: VOTER last_known_addr { host: "127.29.194.61" port: 40955 } attrs { promote: false } } peers { permanent_uuid: "e5603c6f9ea84f1a9510499da5950324" member_type: VOTER last_known_addr { host: "127.29.194.60" port: 44209 } attrs { promote: false } } }
I20260807 15:11:56.163232 8880 sys_catalog.cc:458] T 00000000000000000000000000000000 P e5603c6f9ea84f1a9510499da5950324 [sys.catalog]: This master's current role is: LEADER
I20260807 15:11:56.163363 8888 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8e78c2a81ab3476b8cbeff1833835aeb [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "e5603c6f9ea84f1a9510499da5950324" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "4ebd9481251f4a29ba10f88819ba9ae2" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 39853 } } peers { permanent_uuid: "8e78c2a81ab3476b8cbeff1833835aeb" member_type: VOTER last_known_addr { host: "127.29.194.61" port: 40955 } attrs { promote: false } } peers { permanent_uuid: "e5603c6f9ea84f1a9510499da5950324" member_type: VOTER last_known_addr { host: "127.29.194.60" port: 44209 } attrs { promote: false } } }
I20260807 15:11:56.163456 8888 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8e78c2a81ab3476b8cbeff1833835aeb [sys.catalog]: This master's current role is: FOLLOWER
I20260807 15:11:56.163534 8889 mvcc.cc:205] Tried to move back new op lower bound from 7315929154190274560 to 7315929154037682176. Current Snapshot: MvccSnapshot[applied={T|T < 7315929154190274560}]
I20260807 15:11:56.169791 8902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:56.165825 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.3:39783 (local address 127.0.0.1:47970)
0807 15:11:56.166090 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0807 15:11:56.166098 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0807 15:11:56.166168 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:11:56.166407 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:11:56.166409 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:11:56.166412 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:11:56.166482 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:56.166484 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:56.167516 (+ 1032us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:56.167517 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:56.168715 (+ 1198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:56.168717 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:56.169500 (+ 783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:56.169501 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:56.169569 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:56.169575 (+ 6us) client_negotiation.cc:769] Sending connection context
0807 15:11:56.169633 (+ 58us) client_negotiation.cc:240] Negotiation successful
0807 15:11:56.169688 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":246,"mutex_wait_us":112,"thread_start_us":95,"threads_started":1}
I20260807 15:11:56.169822 8905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:56.166252 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39880 (local address 127.29.194.2:32991)
0807 15:11:56.166564 (+ 312us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:56.166566 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:56.166574 (+ 8us) server_negotiation.cc:407] Connection header received
0807 15:11:56.166603 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:56.166605 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:56.166625 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:56.166705 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:11:56.166803 (+ 98us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:56.167341 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:56.168819 (+ 1478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:56.169418 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:56.169532 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:56.169549 (+ 17us) server_negotiation.cc:1035] Waiting for connection context
0807 15:11:56.169700 (+ 151us) server_negotiation.cc:299] Negotiation successful
0807 15:11:56.169750 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":112,"thread_start_us":68,"threads_started":1}
I20260807 15:11:56.169801 8901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:56.165890 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47970 (local address 127.29.194.3:39783)
0807 15:11:56.166086 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:56.166088 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:56.166272 (+ 184us) server_negotiation.cc:407] Connection header received
0807 15:11:56.166308 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:56.166309 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:56.166321 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:56.166394 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:11:56.166565 (+ 171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:56.167432 (+ 867us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:56.168857 (+ 1425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:56.169362 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:56.169493 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:56.169512 (+ 19us) server_negotiation.cc:1035] Waiting for connection context
0807 15:11:56.169663 (+ 151us) server_negotiation.cc:299] Negotiation successful
0807 15:11:56.169716 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":155,"thread_start_us":109,"threads_started":1}
I20260807 15:11:56.169898 8903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:56.165970 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.2:32991 (local address 127.0.0.1:39880)
0807 15:11:56.166399 (+ 429us) negotiation.cc:107] Waiting for socket to connect
0807 15:11:56.166408 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0807 15:11:56.166447 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:11:56.166704 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:11:56.166704 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:11:56.166706 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:11:56.166755 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:56.166756 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:56.167412 (+ 656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:56.167413 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:56.168715 (+ 1302us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:56.168717 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:56.169541 (+ 824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:56.169542 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:56.169613 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:56.169620 (+ 7us) client_negotiation.cc:769] Sending connection context
0807 15:11:56.169686 (+ 66us) client_negotiation.cc:240] Negotiation successful
0807 15:11:56.169759 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":408,"mutex_wait_us":61,"thread_start_us":114,"threads_started":1}
I20260807 15:11:56.170390 8666 tablet_service.cc:1467] Processing CreateTablet for tablet f4e1e142a369408f938b7503aa5d9594 (DEFAULT_TABLE table=test-table [id=29e7b8c8155e430ca40b75007a698d53]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:11:56.170487 8473 tablet_service.cc:1467] Processing CreateTablet for tablet f4e1e142a369408f938b7503aa5d9594 (DEFAULT_TABLE table=test-table [id=29e7b8c8155e430ca40b75007a698d53]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:11:56.170558 8666 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f4e1e142a369408f938b7503aa5d9594. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:11:56.170799 8473 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f4e1e142a369408f938b7503aa5d9594. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:11:56.170992 8881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:56.165823 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.1:34085 (local address 127.0.0.1:38696)
0807 15:11:56.165894 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0807 15:11:56.165907 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 15:11:56.165972 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:11:56.166532 (+ 560us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:11:56.166533 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:11:56.166534 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:11:56.166586 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:56.166587 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:56.167553 (+ 966us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:56.167555 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:56.168844 (+ 1289us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:56.168851 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:56.170755 (+ 1904us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:56.170757 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:56.170831 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:56.170837 (+ 6us) client_negotiation.cc:769] Sending connection context
0807 15:11:56.170873 (+ 36us) client_negotiation.cc:240] Negotiation successful
0807 15:11:56.170931 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44}
I20260807 15:11:56.171434 8904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:56.166070 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38696 (local address 127.29.194.1:34085)
0807 15:11:56.166232 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:56.166235 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:56.166243 (+ 8us) server_negotiation.cc:407] Connection header received
0807 15:11:56.166277 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:56.166278 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:56.166292 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:56.166508 (+ 216us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:11:56.166654 (+ 146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:56.167450 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:56.169102 (+ 1652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:56.170534 (+ 1432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:56.171194 (+ 660us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:56.171227 (+ 33us) server_negotiation.cc:1035] Waiting for connection context
0807 15:11:56.171253 (+ 26us) server_negotiation.cc:299] Negotiation successful
0807 15:11:56.171354 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127,"thread_start_us":83,"threads_started":1}
I20260807 15:11:56.171718 8278 tablet_service.cc:1467] Processing CreateTablet for tablet f4e1e142a369408f938b7503aa5d9594 (DEFAULT_TABLE table=test-table [id=29e7b8c8155e430ca40b75007a698d53]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:11:56.171854 8278 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f4e1e142a369408f938b7503aa5d9594. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:11:56.172269 8906 tablet_bootstrap.cc:492] T f4e1e142a369408f938b7503aa5d9594 P 9d215d5c47fd4619ad846c7151ad85c1: Bootstrap starting.
I20260807 15:11:56.172695 8906 tablet_bootstrap.cc:654] T f4e1e142a369408f938b7503aa5d9594 P 9d215d5c47fd4619ad846c7151ad85c1: Neither blocks nor log segments found. Creating new log.
I20260807 15:11:56.172875 8907 tablet_bootstrap.cc:492] T f4e1e142a369408f938b7503aa5d9594 P 1a18fd2e4123445cad3e1fa79a7fb96c: Bootstrap starting.
I20260807 15:11:56.173055 8909 tablet_bootstrap.cc:492] T f4e1e142a369408f938b7503aa5d9594 P 10d07cb0cc8840bea5ce1292f24169f3: Bootstrap starting.
I20260807 15:11:56.173249 8907 tablet_bootstrap.cc:654] T f4e1e142a369408f938b7503aa5d9594 P 1a18fd2e4123445cad3e1fa79a7fb96c: Neither blocks nor log segments found. Creating new log.
I20260807 15:11:56.173473 8906 tablet_bootstrap.cc:492] T f4e1e142a369408f938b7503aa5d9594 P 9d215d5c47fd4619ad846c7151ad85c1: No bootstrap required, opened a new log
I20260807 15:11:56.173527 8906 ts_tablet_manager.cc:1397] T f4e1e142a369408f938b7503aa5d9594 P 9d215d5c47fd4619ad846c7151ad85c1: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260807 15:11:56.173656 8906 raft_consensus.cc:348] T f4e1e142a369408f938b7503aa5d9594 P 9d215d5c47fd4619ad846c7151ad85c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10d07cb0cc8840bea5ce1292f24169f3" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 34085 } } peers { permanent_uuid: "9d215d5c47fd4619ad846c7151ad85c1" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 39783 } } peers { permanent_uuid: "1a18fd2e4123445cad3e1fa79a7fb96c" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 32991 } }
I20260807 15:11:56.173738 8906 raft_consensus.cc:374] T f4e1e142a369408f938b7503aa5d9594 P 9d215d5c47fd4619ad846c7151ad85c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:11:56.173759 8906 raft_consensus.cc:729] T f4e1e142a369408f938b7503aa5d9594 P 9d215d5c47fd4619ad846c7151ad85c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9d215d5c47fd4619ad846c7151ad85c1, State: Initialized, Role: FOLLOWER
I20260807 15:11:56.173820 8906 consensus_queue.cc:260] T f4e1e142a369408f938b7503aa5d9594 P 9d215d5c47fd4619ad846c7151ad85c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10d07cb0cc8840bea5ce1292f24169f3" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 34085 } } peers { permanent_uuid: "9d215d5c47fd4619ad846c7151ad85c1" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 39783 } } peers { permanent_uuid: "1a18fd2e4123445cad3e1fa79a7fb96c" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 32991 } }
I20260807 15:11:56.173928 8907 tablet_bootstrap.cc:492] T f4e1e142a369408f938b7503aa5d9594 P 1a18fd2e4123445cad3e1fa79a7fb96c: No bootstrap required, opened a new log
I20260807 15:11:56.173981 8907 ts_tablet_manager.cc:1397] T f4e1e142a369408f938b7503aa5d9594 P 1a18fd2e4123445cad3e1fa79a7fb96c: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260807 15:11:56.174055 8906 ts_tablet_manager.cc:1428] T f4e1e142a369408f938b7503aa5d9594 P 9d215d5c47fd4619ad846c7151ad85c1: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260807 15:11:56.174118 8907 raft_consensus.cc:348] T f4e1e142a369408f938b7503aa5d9594 P 1a18fd2e4123445cad3e1fa79a7fb96c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10d07cb0cc8840bea5ce1292f24169f3" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 34085 } } peers { permanent_uuid: "9d215d5c47fd4619ad846c7151ad85c1" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 39783 } } peers { permanent_uuid: "1a18fd2e4123445cad3e1fa79a7fb96c" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 32991 } }
I20260807 15:11:56.174196 8907 raft_consensus.cc:374] T f4e1e142a369408f938b7503aa5d9594 P 1a18fd2e4123445cad3e1fa79a7fb96c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:11:56.174218 8907 raft_consensus.cc:729] T f4e1e142a369408f938b7503aa5d9594 P 1a18fd2e4123445cad3e1fa79a7fb96c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1a18fd2e4123445cad3e1fa79a7fb96c, State: Initialized, Role: FOLLOWER
I20260807 15:11:56.174263 8907 consensus_queue.cc:260] T f4e1e142a369408f938b7503aa5d9594 P 1a18fd2e4123445cad3e1fa79a7fb96c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10d07cb0cc8840bea5ce1292f24169f3" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 34085 } } peers { permanent_uuid: "9d215d5c47fd4619ad846c7151ad85c1" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 39783 } } peers { permanent_uuid: "1a18fd2e4123445cad3e1fa79a7fb96c" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 32991 } }
I20260807 15:11:56.174422 8879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:56.174212 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:39853 (local address 127.29.194.3:48955)
0807 15:11:56.174358 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0807 15:11:56.174391 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.194.62:39853: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127}
I20260807 15:11:56.174451 8907 ts_tablet_manager.cc:1428] T f4e1e142a369408f938b7503aa5d9594 P 1a18fd2e4123445cad3e1fa79a7fb96c: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260807 15:11:56.174804 8878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:56.174662 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:39853 (local address 127.29.194.2:36333)
0807 15:11:56.174750 (+ 88us) negotiation.cc:107] Waiting for socket to connect
0807 15:11:56.174777 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.194.62:39853: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260807 15:11:56.175031 8879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:56.174683 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:39853 (local address 127.29.194.3:43735)
0807 15:11:56.174977 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0807 15:11:56.175006 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.194.62:39853: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":280}
I20260807 15:11:56.175051 8909 tablet_bootstrap.cc:654] T f4e1e142a369408f938b7503aa5d9594 P 10d07cb0cc8840bea5ce1292f24169f3: Neither blocks nor log segments found. Creating new log.
I20260807 15:11:56.175105 8756 heartbeater.cc:503] Master 127.29.194.60:44209 was elected leader, sending a full tablet report...
I20260807 15:11:56.175606 8570 heartbeater.cc:503] Master 127.29.194.60:44209 was elected leader, sending a full tablet report...
I20260807 15:11:56.175900 8878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:56.175803 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:39853 (local address 127.29.194.2:45003)
0807 15:11:56.175842 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0807 15:11:56.175875 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.194.62:39853: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":20}
I20260807 15:11:56.175917 8909 tablet_bootstrap.cc:492] T f4e1e142a369408f938b7503aa5d9594 P 10d07cb0cc8840bea5ce1292f24169f3: No bootstrap required, opened a new log
I20260807 15:11:56.175979 8909 ts_tablet_manager.cc:1397] T f4e1e142a369408f938b7503aa5d9594 P 10d07cb0cc8840bea5ce1292f24169f3: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260807 15:11:56.176106 8909 raft_consensus.cc:348] T f4e1e142a369408f938b7503aa5d9594 P 10d07cb0cc8840bea5ce1292f24169f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10d07cb0cc8840bea5ce1292f24169f3" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 34085 } } peers { permanent_uuid: "9d215d5c47fd4619ad846c7151ad85c1" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 39783 } } peers { permanent_uuid: "1a18fd2e4123445cad3e1fa79a7fb96c" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 32991 } }
I20260807 15:11:56.176178 8909 raft_consensus.cc:374] T f4e1e142a369408f938b7503aa5d9594 P 10d07cb0cc8840bea5ce1292f24169f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:11:56.176192 8909 raft_consensus.cc:729] T f4e1e142a369408f938b7503aa5d9594 P 10d07cb0cc8840bea5ce1292f24169f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 10d07cb0cc8840bea5ce1292f24169f3, State: Initialized, Role: FOLLOWER
I20260807 15:11:56.176229 8909 consensus_queue.cc:260] T f4e1e142a369408f938b7503aa5d9594 P 10d07cb0cc8840bea5ce1292f24169f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10d07cb0cc8840bea5ce1292f24169f3" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 34085 } } peers { permanent_uuid: "9d215d5c47fd4619ad846c7151ad85c1" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 39783 } } peers { permanent_uuid: "1a18fd2e4123445cad3e1fa79a7fb96c" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 32991 } }
I20260807 15:11:56.176386 8909 ts_tablet_manager.cc:1428] T f4e1e142a369408f938b7503aa5d9594 P 10d07cb0cc8840bea5ce1292f24169f3: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260807 15:11:56.176780 8875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:56.176677 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:39853 (local address 127.29.194.1:49889)
0807 15:11:56.176719 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0807 15:11:56.176751 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.194.62:39853: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
W20260807 15:11:56.208815 8762 tablet.cc:2369] T f4e1e142a369408f938b7503aa5d9594 P 9d215d5c47fd4619ad846c7151ad85c1: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 15:11:56.221231 8911 raft_consensus.cc:482] T f4e1e142a369408f938b7503aa5d9594 P 9d215d5c47fd4619ad846c7151ad85c1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 15:11:56.221400 8911 raft_consensus.cc:504] T f4e1e142a369408f938b7503aa5d9594 P 9d215d5c47fd4619ad846c7151ad85c1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10d07cb0cc8840bea5ce1292f24169f3" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 34085 } } peers { permanent_uuid: "9d215d5c47fd4619ad846c7151ad85c1" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 39783 } } peers { permanent_uuid: "1a18fd2e4123445cad3e1fa79a7fb96c" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 32991 } }
I20260807 15:11:56.221971 8911 leader_election.cc:288] T f4e1e142a369408f938b7503aa5d9594 P 9d215d5c47fd4619ad846c7151ad85c1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 10d07cb0cc8840bea5ce1292f24169f3 (127.29.194.1:34085), 1a18fd2e4123445cad3e1fa79a7fb96c (127.29.194.2:32991)
I20260807 15:11:56.226687 8904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:56.222346 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.194.3:38177 (local address 127.29.194.1:34085)
0807 15:11:56.222407 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:56.222409 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:56.222426 (+ 17us) server_negotiation.cc:407] Connection header received
0807 15:11:56.222568 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:56.222569 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:56.222578 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:56.222655 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:11:56.222930 (+ 275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:56.223706 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:56.225612 (+ 1906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:56.226201 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:56.226292 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:56.226303 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0807 15:11:56.226570 (+ 267us) server_negotiation.cc:299] Negotiation successful
0807 15:11:56.226613 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260807 15:11:56.226687 8905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:56.222446 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.194.3:49247 (local address 127.29.194.2:32991)
0807 15:11:56.222517 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:56.222519 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:56.222666 (+ 147us) server_negotiation.cc:407] Connection header received
0807 15:11:56.222697 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:56.222699 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:56.222713 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:56.222761 (+ 48us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:11:56.222941 (+ 180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:56.223729 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:56.225613 (+ 1884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:56.226201 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:56.226292 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:56.226303 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0807 15:11:56.226563 (+ 260us) server_negotiation.cc:299] Negotiation successful
0807 15:11:56.226608 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260807 15:11:56.226724 8879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:56.222211 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.1:34085 (local address 127.29.194.3:38177)
0807 15:11:56.222294 (+ 83us) negotiation.cc:107] Waiting for socket to connect
0807 15:11:56.222311 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 15:11:56.222430 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:11:56.222720 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:11:56.222721 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:11:56.222724 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:11:56.222837 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:56.222839 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:56.223863 (+ 1024us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:56.223865 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:56.225501 (+ 1636us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:56.225507 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:56.226367 (+ 860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:56.226369 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:56.226460 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:56.226467 (+ 7us) client_negotiation.cc:769] Sending connection context
0807 15:11:56.226559 (+ 92us) client_negotiation.cc:240] Negotiation successful
0807 15:11:56.226604 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":51,"mutex_wait_us":79}
I20260807 15:11:56.226730 8916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:56.222331 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.2:32991 (local address 127.29.194.3:49247)
0807 15:11:56.222558 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0807 15:11:56.222567 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0807 15:11:56.222623 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:11:56.222778 (+ 155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:11:56.222778 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:11:56.222779 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:11:56.222862 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:56.222864 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:56.223863 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:56.223865 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:56.225501 (+ 1636us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:56.225507 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:56.226346 (+ 839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:56.226348 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:56.226460 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:56.226469 (+ 9us) client_negotiation.cc:769] Sending connection context
0807 15:11:56.226548 (+ 79us) client_negotiation.cc:240] Negotiation successful
0807 15:11:56.226604 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":204,"mutex_wait_us":53,"thread_start_us":138,"threads_started":1}
I20260807 15:11:56.227061 8502 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4e1e142a369408f938b7503aa5d9594" candidate_uuid: "9d215d5c47fd4619ad846c7151ad85c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1a18fd2e4123445cad3e1fa79a7fb96c" is_pre_election: true
I20260807 15:11:56.227160 8502 raft_consensus.cc:2454] T f4e1e142a369408f938b7503aa5d9594 P 1a18fd2e4123445cad3e1fa79a7fb96c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9d215d5c47fd4619ad846c7151ad85c1 in term 0.
I20260807 15:11:56.227257 8308 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4e1e142a369408f938b7503aa5d9594" candidate_uuid: "9d215d5c47fd4619ad846c7151ad85c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "10d07cb0cc8840bea5ce1292f24169f3" is_pre_election: true
I20260807 15:11:56.227365 8600 leader_election.cc:302] T f4e1e142a369408f938b7503aa5d9594 P 9d215d5c47fd4619ad846c7151ad85c1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1a18fd2e4123445cad3e1fa79a7fb96c, 9d215d5c47fd4619ad846c7151ad85c1; no voters:
I20260807 15:11:56.227418 8308 raft_consensus.cc:2454] T f4e1e142a369408f938b7503aa5d9594 P 10d07cb0cc8840bea5ce1292f24169f3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9d215d5c47fd4619ad846c7151ad85c1 in term 0.
I20260807 15:11:56.227538 8911 raft_consensus.cc:2781] T f4e1e142a369408f938b7503aa5d9594 P 9d215d5c47fd4619ad846c7151ad85c1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 15:11:56.227596 8911 raft_consensus.cc:482] T f4e1e142a369408f938b7503aa5d9594 P 9d215d5c47fd4619ad846c7151ad85c1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 15:11:56.227612 8911 raft_consensus.cc:3037] T f4e1e142a369408f938b7503aa5d9594 P 9d215d5c47fd4619ad846c7151ad85c1 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:11:56.228343 8911 raft_consensus.cc:504] T f4e1e142a369408f938b7503aa5d9594 P 9d215d5c47fd4619ad846c7151ad85c1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10d07cb0cc8840bea5ce1292f24169f3" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 34085 } } peers { permanent_uuid: "9d215d5c47fd4619ad846c7151ad85c1" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 39783 } } peers { permanent_uuid: "1a18fd2e4123445cad3e1fa79a7fb96c" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 32991 } }
I20260807 15:11:56.228489 8911 leader_election.cc:288] T f4e1e142a369408f938b7503aa5d9594 P 9d215d5c47fd4619ad846c7151ad85c1 [CANDIDATE]: Term 1 election: Requested vote from peers 10d07cb0cc8840bea5ce1292f24169f3 (127.29.194.1:34085), 1a18fd2e4123445cad3e1fa79a7fb96c (127.29.194.2:32991)
I20260807 15:11:56.228682 8308 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4e1e142a369408f938b7503aa5d9594" candidate_uuid: "9d215d5c47fd4619ad846c7151ad85c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "10d07cb0cc8840bea5ce1292f24169f3"
I20260807 15:11:56.228732 8502 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4e1e142a369408f938b7503aa5d9594" candidate_uuid: "9d215d5c47fd4619ad846c7151ad85c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1a18fd2e4123445cad3e1fa79a7fb96c"
I20260807 15:11:56.228768 8308 raft_consensus.cc:3037] T f4e1e142a369408f938b7503aa5d9594 P 10d07cb0cc8840bea5ce1292f24169f3 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:11:56.228798 8502 raft_consensus.cc:3037] T f4e1e142a369408f938b7503aa5d9594 P 1a18fd2e4123445cad3e1fa79a7fb96c [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:11:56.229390 8502 raft_consensus.cc:2454] T f4e1e142a369408f938b7503aa5d9594 P 1a18fd2e4123445cad3e1fa79a7fb96c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9d215d5c47fd4619ad846c7151ad85c1 in term 1.
I20260807 15:11:56.229514 8308 raft_consensus.cc:2454] T f4e1e142a369408f938b7503aa5d9594 P 10d07cb0cc8840bea5ce1292f24169f3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9d215d5c47fd4619ad846c7151ad85c1 in term 1.
I20260807 15:11:56.229585 8600 leader_election.cc:302] T f4e1e142a369408f938b7503aa5d9594 P 9d215d5c47fd4619ad846c7151ad85c1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1a18fd2e4123445cad3e1fa79a7fb96c, 9d215d5c47fd4619ad846c7151ad85c1; no voters:
I20260807 15:11:56.229676 8911 raft_consensus.cc:2781] T f4e1e142a369408f938b7503aa5d9594 P 9d215d5c47fd4619ad846c7151ad85c1 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:11:56.229844 8911 raft_consensus.cc:686] T f4e1e142a369408f938b7503aa5d9594 P 9d215d5c47fd4619ad846c7151ad85c1 [term 1 LEADER]: Becoming Leader. State: Replica: 9d215d5c47fd4619ad846c7151ad85c1, State: Running, Role: LEADER
I20260807 15:11:56.229914 8911 consensus_queue.cc:237] T f4e1e142a369408f938b7503aa5d9594 P 9d215d5c47fd4619ad846c7151ad85c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10d07cb0cc8840bea5ce1292f24169f3" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 34085 } } peers { permanent_uuid: "9d215d5c47fd4619ad846c7151ad85c1" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 39783 } } peers { permanent_uuid: "1a18fd2e4123445cad3e1fa79a7fb96c" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 32991 } }
I20260807 15:11:56.230229 8916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:56.230129 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:39853 (local address 127.29.194.3:50589)
0807 15:11:56.230163 (+ 34us) negotiation.cc:107] Waiting for socket to connect
0807 15:11:56.230198 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.194.62:39853: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":17}
I20260807 15:11:56.230762 8125 catalog_manager.cc:5495] T f4e1e142a369408f938b7503aa5d9594 P 9d215d5c47fd4619ad846c7151ad85c1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9d215d5c47fd4619ad846c7151ad85c1 (127.29.194.3). New cstate: current_term: 1 leader_uuid: "9d215d5c47fd4619ad846c7151ad85c1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10d07cb0cc8840bea5ce1292f24169f3" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 34085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9d215d5c47fd4619ad846c7151ad85c1" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 39783 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1a18fd2e4123445cad3e1fa79a7fb96c" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 32991 } health_report { overall_health: UNKNOWN } } }
I20260807 15:11:56.247030 8901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:56.243918 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47972 (local address 127.29.194.3:39783)
0807 15:11:56.244009 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:56.244011 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:56.244024 (+ 13us) server_negotiation.cc:407] Connection header received
0807 15:11:56.244081 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:56.244083 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:56.244111 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:56.244207 (+ 96us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 15:11:56.244451 (+ 244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:56.244967 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:56.246074 (+ 1107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:56.246281 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:56.246352 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:56.246618 (+ 266us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 15:11:56.246740 (+ 122us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 15:11:56.246812 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0807 15:11:56.246916 (+ 104us) server_negotiation.cc:299] Negotiation successful
0807 15:11:56.246943 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260807 15:11:56.248559 8502 raft_consensus.cc:1261] T f4e1e142a369408f938b7503aa5d9594 P 1a18fd2e4123445cad3e1fa79a7fb96c [term 1 FOLLOWER]: Refusing update from remote peer 9d215d5c47fd4619ad846c7151ad85c1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:11:56.248638 8308 raft_consensus.cc:1261] T f4e1e142a369408f938b7503aa5d9594 P 10d07cb0cc8840bea5ce1292f24169f3 [term 1 FOLLOWER]: Refusing update from remote peer 9d215d5c47fd4619ad846c7151ad85c1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:11:56.248914 8911 consensus_queue.cc:1035] T f4e1e142a369408f938b7503aa5d9594 P 9d215d5c47fd4619ad846c7151ad85c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1a18fd2e4123445cad3e1fa79a7fb96c" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 32991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:11:56.249068 8917 consensus_queue.cc:1035] T f4e1e142a369408f938b7503aa5d9594 P 9d215d5c47fd4619ad846c7151ad85c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d07cb0cc8840bea5ce1292f24169f3" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 34085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:11:56.249571 8374 heartbeater.cc:503] Master 127.29.194.60:44209 was elected leader, sending a full tablet report...
I20260807 15:11:56.250036 8878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:56.249920 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:39853 (local address 127.29.194.2:43903)
0807 15:11:56.249969 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0807 15:11:56.250002 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.194.62:39853: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260807 15:11:56.250200 8875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:56.250069 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:39853 (local address 127.29.194.1:38525)
0807 15:11:56.250142 (+ 73us) negotiation.cc:107] Waiting for socket to connect
0807 15:11:56.250174 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.194.62:39853: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260807 15:11:56.251763 8875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:56.251536 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:39853 (local address 127.29.194.1:37189)
0807 15:11:56.251703 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0807 15:11:56.251735 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.194.62:39853: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":150}
I20260807 15:11:56.251809 8916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:56.251327 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:39853 (local address 127.29.194.3:47027)
0807 15:11:56.251618 (+ 291us) negotiation.cc:107] Waiting for socket to connect
0807 15:11:56.251775 (+ 157us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.194.62:39853: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":272}
I20260807 15:11:56.252107 8879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:56.251995 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:39853 (local address 127.29.194.3:50947)
0807 15:11:56.252047 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0807 15:11:56.252079 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.194.62:39853: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260807 15:11:56.252090 8925 mvcc.cc:205] Tried to move back new op lower bound from 7315929154551390208 to 7315929154478411776. Current Snapshot: MvccSnapshot[applied={T|T < 7315929154551390208}]
I20260807 15:11:56.253628 8878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:56.253527 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:39853 (local address 127.29.194.2:58603)
0807 15:11:56.253569 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0807 15:11:56.253594 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.194.62:39853: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
I20260807 15:11:56.254211 8125 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33830:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 15:11:56.254308 8125 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33830:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 15:11:56.255780 8125 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e5603c6f9ea84f1a9510499da5950324: Sending DeleteTablet for 3 replicas of tablet f4e1e142a369408f938b7503aa5d9594
I20260807 15:11:56.256109 8473 tablet_service.cc:1514] Processing DeleteTablet for tablet f4e1e142a369408f938b7503aa5d9594 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:11:56 UTC) from {username='oryx', principal='oryx/127.29.194.60@KRBTEST.COM'} at 127.0.0.1:39880
I20260807 15:11:56.256137 8278 tablet_service.cc:1514] Processing DeleteTablet for tablet f4e1e142a369408f938b7503aa5d9594 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:11:56 UTC) from {username='oryx', principal='oryx/127.29.194.60@KRBTEST.COM'} at 127.0.0.1:38696
I20260807 15:11:56.256132 8666 tablet_service.cc:1514] Processing DeleteTablet for tablet f4e1e142a369408f938b7503aa5d9594 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:11:56 UTC) from {username='oryx', principal='oryx/127.29.194.60@KRBTEST.COM'} at 127.0.0.1:47970
I20260807 15:11:56.256333 8934 tablet_replica.cc:330] T f4e1e142a369408f938b7503aa5d9594 P 10d07cb0cc8840bea5ce1292f24169f3: stopping tablet replica
I20260807 15:11:56.256387 8934 raft_consensus.cc:2229] T f4e1e142a369408f938b7503aa5d9594 P 10d07cb0cc8840bea5ce1292f24169f3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 15:11:56.256433 8934 raft_consensus.cc:2258] T f4e1e142a369408f938b7503aa5d9594 P 10d07cb0cc8840bea5ce1292f24169f3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:11:56.256531 8933 tablet_replica.cc:330] T f4e1e142a369408f938b7503aa5d9594 P 1a18fd2e4123445cad3e1fa79a7fb96c: stopping tablet replica
I20260807 15:11:56.256587 8933 raft_consensus.cc:2229] T f4e1e142a369408f938b7503aa5d9594 P 1a18fd2e4123445cad3e1fa79a7fb96c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 15:11:56.256637 8933 raft_consensus.cc:2258] T f4e1e142a369408f938b7503aa5d9594 P 1a18fd2e4123445cad3e1fa79a7fb96c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:11:56.256747 8935 tablet_replica.cc:330] T f4e1e142a369408f938b7503aa5d9594 P 9d215d5c47fd4619ad846c7151ad85c1: stopping tablet replica
I20260807 15:11:56.256806 8935 raft_consensus.cc:2229] T f4e1e142a369408f938b7503aa5d9594 P 9d215d5c47fd4619ad846c7151ad85c1 [term 1 LEADER]: Raft consensus shutting down.
I20260807 15:11:56.256902 30472 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7fZSIL/build/release/bin/kudu with pid 8073
I20260807 15:11:56.261616 8933 ts_tablet_manager.cc:1905] T f4e1e142a369408f938b7503aa5d9594 P 1a18fd2e4123445cad3e1fa79a7fb96c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:11:56.264148 8933 ts_tablet_manager.cc:1918] T f4e1e142a369408f938b7503aa5d9594 P 1a18fd2e4123445cad3e1fa79a7fb96c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 15:11:56.264251 8933 log.cc:1196] T f4e1e142a369408f938b7503aa5d9594 P 1a18fd2e4123445cad3e1fa79a7fb96c: Deleting WAL directory at /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115480851507-30472-0/minicluster-data/ts-1/wal/wals/f4e1e142a369408f938b7503aa5d9594
I20260807 15:11:56.264498 8933 ts_tablet_manager.cc:1939] T f4e1e142a369408f938b7503aa5d9594 P 1a18fd2e4123445cad3e1fa79a7fb96c: Deleting consensus metadata
I20260807 15:11:56.264743 8935 raft_consensus.cc:2258] T f4e1e142a369408f938b7503aa5d9594 P 9d215d5c47fd4619ad846c7151ad85c1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:11:56.265786 8105 catalog_manager.cc:4841] TS 1a18fd2e4123445cad3e1fa79a7fb96c (127.29.194.2:32991): tablet f4e1e142a369408f938b7503aa5d9594 (table test-table [id=29e7b8c8155e430ca40b75007a698d53]) successfully deleted
I20260807 15:11:56.266942 8935 ts_tablet_manager.cc:1905] T f4e1e142a369408f938b7503aa5d9594 P 9d215d5c47fd4619ad846c7151ad85c1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:11:56.269369 8935 ts_tablet_manager.cc:1918] T f4e1e142a369408f938b7503aa5d9594 P 9d215d5c47fd4619ad846c7151ad85c1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 15:11:56.269636 8935 log.cc:1196] T f4e1e142a369408f938b7503aa5d9594 P 9d215d5c47fd4619ad846c7151ad85c1: Deleting WAL directory at /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115480851507-30472-0/minicluster-data/ts-2/wal/wals/f4e1e142a369408f938b7503aa5d9594
I20260807 15:11:56.269970 8935 ts_tablet_manager.cc:1939] T f4e1e142a369408f938b7503aa5d9594 P 9d215d5c47fd4619ad846c7151ad85c1: Deleting consensus metadata
I20260807 15:11:56.270443 8103 catalog_manager.cc:4841] TS 9d215d5c47fd4619ad846c7151ad85c1 (127.29.194.3:39783): tablet f4e1e142a369408f938b7503aa5d9594 (table test-table [id=29e7b8c8155e430ca40b75007a698d53]) successfully deleted
W20260807 15:11:56.271894 8881 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:11:56.271655 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.1:34085 (local address 127.0.0.1:38708)
0807 15:11:56.271713 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0807 15:11:56.271727 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 15:11:56.271784 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:11:56.271852 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.194.1:34085: BlockingWrite error: write error: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":36}
I20260807 15:11:56.272125 30472 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7fZSIL/build/release/bin/kudu with pid 8383
W20260807 15:11:56.272187 8105 catalog_manager.cc:4584] TS 10d07cb0cc8840bea5ce1292f24169f3 (127.29.194.1:34085): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet f4e1e142a369408f938b7503aa5d9594: Network error: Client connection negotiation failed: client connection to 127.29.194.1:34085: BlockingWrite error: write error: Connection reset by peer (error 104)
I20260807 15:11:56.283955 30472 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7fZSIL/build/release/bin/kudu with pid 8578
I20260807 15:11:56.302007 30472 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7fZSIL/build/release/bin/kudu with pid 7668
I20260807 15:11:56.314500 30472 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7fZSIL/build/release/bin/kudu with pid 7936
2026-08-07T15:11:56Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (6538 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115480851507-30472-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 15:11:56 dist-test-slave-42z9 krb5kdc[8938](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 15:11:56 dist-test-slave-42z9 krb5kdc[8938](info): set up 2 sockets
Aug 07 15:11:56 dist-test-slave-42z9 krb5kdc[8938](info): commencing operation
krb5kdc: starting...
W20260807 15:11:57.491593 30472 mini_kdc.cc:121] Time spent starting KDC: real 1.148s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115480851507-30472-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115480851507-30472-0/krb5kdc/test-user.keytab.
Aug 07 15:11:57 dist-test-slave-42z9 krb5kdc[8938](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115517, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T15:11:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T15:11:57Z Disabled control of system clock
WARNING: no policy specified for oryx/127.29.194.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.29.194.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.29.194.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115480851507-30472-0/krb5kdc/oryx_127.29.194.62.keytab.
Entry for principal oryx/127.29.194.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115480851507-30472-0/krb5kdc/oryx_127.29.194.62.keytab.
I20260807 15:11:57.641914 30472 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7fZSIL/build/release/bin/kudu
/tmp/dist-test-task7fZSIL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115480851507-30472-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115480851507-30472-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115480851507-30472-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115480851507-30472-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.194.62:34237
--webserver_interface=127.29.194.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115480851507-30472-0/krb5kdc/oryx_127.29.194.62.keytab
--principal=oryx/127.29.194.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.194.20:44921
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.194.62:34237
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115480851507-30472-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115480851507-30472-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115480851507-30472-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:11:57.733255 8956 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:11:57.733518 8956 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:11:57.733551 8956 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:11:57.734863 8956 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:11:57.734920 8956 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260807 15:11:57.734932 8956 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:11:57.734943 8956 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:11:57.734956 8956 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:11:57.734966 8956 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:11:57.736716 8956 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.194.20:44921
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115480851507-30472-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115480851507-30472-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.194.62:34237
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115480851507-30472-0/krb5kdc/oryx_127.29.194.62.keytab
--principal=oryx/127.29.194.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.194.62:34237
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115480851507-30472-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.194.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8956
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115480851507-30472-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:11:57.736984 8956 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:11:57.737265 8956 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:11:57.743294 8964 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:11:57.743355 8965 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:11:57.743341 8956 server_base.cc:1034] running on GCE node
W20260807 15:11:57.743420 8967 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:11:57.743709 8956 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:11:57.744046 8956 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:11:57.745261 8956 hybrid_clock.cc:648] HybridClock initialized: now 1786115517745213 us; error 58 us; skew 500 ppm
Aug 07 15:11:57 dist-test-slave-42z9 krb5kdc[8938](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115517, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.194.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:11:57.748605 8956 init.cc:377] Logged in from keytab as oryx/127.29.194.62@KRBTEST.COM (short username oryx)
I20260807 15:11:57.749138 8956 webserver.cc:460] Webserver started at http://127.29.194.62:42241/ using document root <none> and password file <none>
I20260807 15:11:57.749411 8956 fs_manager.cc:359] Metadata directory not provided
I20260807 15:11:57.749502 8956 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:11:57.749646 8956 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:11:57.750775 8956 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115480851507-30472-0/minicluster-data/master-0/data/instance:
uuid: "f743aff5796d4ca99c67bbf7b9b722d6"
format_stamp: "Formatted at 2026-08-07 15:11:57 on dist-test-slave-42z9"
server_key: "55f9dc61c666b4c970511ee60b2ba127"
server_key_iv: "e46bcc27245045754b232e4e5b90f44f"
server_key_version: "encryptionkey@0"
I20260807 15:11:57.751150 8956 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115480851507-30472-0/minicluster-data/master-0/wal/instance:
uuid: "f743aff5796d4ca99c67bbf7b9b722d6"
format_stamp: "Formatted at 2026-08-07 15:11:57 on dist-test-slave-42z9"
server_key: "55f9dc61c666b4c970511ee60b2ba127"
server_key_iv: "e46bcc27245045754b232e4e5b90f44f"
server_key_version: "encryptionkey@0"
I20260807 15:11:57.752576 8956 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 15:11:57.753424 8980 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:11:57.753679 8956 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:11:57.753782 8956 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115480851507-30472-0/minicluster-data/master-0/data,/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115480851507-30472-0/minicluster-data/master-0/wal
uuid: "f743aff5796d4ca99c67bbf7b9b722d6"
format_stamp: "Formatted at 2026-08-07 15:11:57 on dist-test-slave-42z9"
server_key: "55f9dc61c666b4c970511ee60b2ba127"
server_key_iv: "e46bcc27245045754b232e4e5b90f44f"
server_key_version: "encryptionkey@0"
I20260807 15:11:57.753839 8956 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115480851507-30472-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115480851507-30472-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115480851507-30472-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:11:57.789515 8956 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:11:57.792567 8956 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:11:57.792786 8956 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:11:57.799067 8956 rpc_server.cc:307] RPC server started. Bound to: 127.29.194.62:34237
I20260807 15:11:57.799126 9082 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.194.62:34237 every 8 connection(s)
I20260807 15:11:57.799522 8956 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115480851507-30472-0/minicluster-data/master-0/data/info.pb
I20260807 15:11:57.800307 9084 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:11:57.800442 30472 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7fZSIL/build/release/bin/kudu as pid 8956
I20260807 15:11:57.800562 30472 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115480851507-30472-0/minicluster-data/master-0/wal/instance
I20260807 15:11:57.800767 30472 external_mini_cluster.cc:1428] Setting key 7fd3f64bec4c9ee35a7b34cc21018b0d
I20260807 15:11:57.803646 9084 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f743aff5796d4ca99c67bbf7b9b722d6: Bootstrap starting.
I20260807 15:11:57.804343 9084 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f743aff5796d4ca99c67bbf7b9b722d6: Neither blocks nor log segments found. Creating new log.
I20260807 15:11:57.804661 9084 log.cc:824] T 00000000000000000000000000000000 P f743aff5796d4ca99c67bbf7b9b722d6: Log is configured to *not* fsync() on all Append() calls
I20260807 15:11:57.805541 9084 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f743aff5796d4ca99c67bbf7b9b722d6: No bootstrap required, opened a new log
I20260807 15:11:57.808130 9084 raft_consensus.cc:348] T 00000000000000000000000000000000 P f743aff5796d4ca99c67bbf7b9b722d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f743aff5796d4ca99c67bbf7b9b722d6" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 34237 } }
I20260807 15:11:57.808349 9084 raft_consensus.cc:374] T 00000000000000000000000000000000 P f743aff5796d4ca99c67bbf7b9b722d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:11:57.808391 9084 raft_consensus.cc:729] T 00000000000000000000000000000000 P f743aff5796d4ca99c67bbf7b9b722d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f743aff5796d4ca99c67bbf7b9b722d6, State: Initialized, Role: FOLLOWER
I20260807 15:11:57.808537 9084 consensus_queue.cc:260] T 00000000000000000000000000000000 P f743aff5796d4ca99c67bbf7b9b722d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f743aff5796d4ca99c67bbf7b9b722d6" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 34237 } }
I20260807 15:11:57.808619 9084 raft_consensus.cc:388] T 00000000000000000000000000000000 P f743aff5796d4ca99c67bbf7b9b722d6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:11:57.808651 9084 raft_consensus.cc:482] T 00000000000000000000000000000000 P f743aff5796d4ca99c67bbf7b9b722d6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:11:57.808698 9084 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f743aff5796d4ca99c67bbf7b9b722d6 [term 0 FOLLOWER]: Advancing to term 1
Aug 07 15:11:57 dist-test-slave-42z9 krb5kdc[8938](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115517, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.29.194.62@KRBTEST.COM
I20260807 15:11:57.809396 9084 raft_consensus.cc:504] T 00000000000000000000000000000000 P f743aff5796d4ca99c67bbf7b9b722d6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f743aff5796d4ca99c67bbf7b9b722d6" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 34237 } }
I20260807 15:11:57.809540 9084 leader_election.cc:302] T 00000000000000000000000000000000 P f743aff5796d4ca99c67bbf7b9b722d6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f743aff5796d4ca99c67bbf7b9b722d6; no voters:
I20260807 15:11:57.809725 9084 leader_election.cc:288] T 00000000000000000000000000000000 P f743aff5796d4ca99c67bbf7b9b722d6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 15:11:57.809774 9091 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f743aff5796d4ca99c67bbf7b9b722d6 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:11:57.809901 9091 raft_consensus.cc:686] T 00000000000000000000000000000000 P f743aff5796d4ca99c67bbf7b9b722d6 [term 1 LEADER]: Becoming Leader. State: Replica: f743aff5796d4ca99c67bbf7b9b722d6, State: Running, Role: LEADER
I20260807 15:11:57.810002 9091 consensus_queue.cc:237] T 00000000000000000000000000000000 P f743aff5796d4ca99c67bbf7b9b722d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f743aff5796d4ca99c67bbf7b9b722d6" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 34237 } }
I20260807 15:11:57.810045 9084 sys_catalog.cc:564] T 00000000000000000000000000000000 P f743aff5796d4ca99c67bbf7b9b722d6 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:11:57.810384 9091 sys_catalog.cc:455] T 00000000000000000000000000000000 P f743aff5796d4ca99c67bbf7b9b722d6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f743aff5796d4ca99c67bbf7b9b722d6. Latest consensus state: current_term: 1 leader_uuid: "f743aff5796d4ca99c67bbf7b9b722d6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f743aff5796d4ca99c67bbf7b9b722d6" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 34237 } } }
I20260807 15:11:57.810477 9091 sys_catalog.cc:458] T 00000000000000000000000000000000 P f743aff5796d4ca99c67bbf7b9b722d6 [sys.catalog]: This master's current role is: LEADER
I20260807 15:11:57.810390 9093 sys_catalog.cc:455] T 00000000000000000000000000000000 P f743aff5796d4ca99c67bbf7b9b722d6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f743aff5796d4ca99c67bbf7b9b722d6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f743aff5796d4ca99c67bbf7b9b722d6" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 34237 } } }
I20260807 15:11:57.810526 9093 sys_catalog.cc:458] T 00000000000000000000000000000000 P f743aff5796d4ca99c67bbf7b9b722d6 [sys.catalog]: This master's current role is: LEADER
I20260807 15:11:57.811570 9098 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:11:57.812472 9098 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:11:57.814313 9098 catalog_manager.cc:1347] Generated new cluster ID: 5a236154e01f4e86955a33d8761c795c
I20260807 15:11:57.814349 9088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:57.801559 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56708 (local address 127.29.194.62:34237)
0807 15:11:57.801850 (+ 291us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:57.801861 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:57.801896 (+ 35us) server_negotiation.cc:407] Connection header received
0807 15:11:57.802526 (+ 630us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:57.802536 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:57.802608 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:57.802841 (+ 233us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:11:57.803663 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:57.804791 (+ 1128us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:57.805904 (+ 1113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:57.806524 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:57.806711 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:57.810102 (+ 3391us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:11:57.810108 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:11:57.810108 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:11:57.810124 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:11:57.812470 (+ 2346us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:57.812977 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:57.812980 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:57.812980 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:57.813106 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:57.813317 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:57.813318 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:57.813318 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:57.813684 (+ 366us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:11:57.813833 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0807 15:11:57.813997 (+ 164us) server_negotiation.cc:299] Negotiation successful
0807 15:11:57.814153 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":194,"thread_start_us":136,"threads_started":1}
I20260807 15:11:57.814368 9098 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:11:57.840461 9098 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:11:57.841521 9098 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:11:57.845091 9098 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f743aff5796d4ca99c67bbf7b9b722d6: Generated new TSK 0
I20260807 15:11:57.845293 9098 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.29.194.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.29.194.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.29.194.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115480851507-30472-0/krb5kdc/oryx_127.29.194.1.keytab.
Entry for principal oryx/127.29.194.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115480851507-30472-0/krb5kdc/oryx_127.29.194.1.keytab.
I20260807 15:11:57.881160 30472 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7fZSIL/build/release/bin/kudu
/tmp/dist-test-task7fZSIL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115480851507-30472-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115480851507-30472-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115480851507-30472-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115480851507-30472-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.194.1:0
--local_ip_for_outbound_sockets=127.29.194.1
--webserver_interface=127.29.194.1
--webserver_port=0
--tserver_master_addrs=127.29.194.62:34237
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115480851507-30472-0/krb5kdc/oryx_127.29.194.1.keytab
--principal=oryx/127.29.194.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.194.20:44921
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115480851507-30472-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115480851507-30472-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115480851507-30472-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:11:57.978603 9122 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:11:57.978849 9122 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:11:57.978878 9122 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:11:57.980218 9122 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:11:57.980278 9122 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:11:57.980300 9122 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 15:11:57.980319 9122 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.194.1
I20260807 15:11:57.982113 9122 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.194.20:44921
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115480851507-30472-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115480851507-30472-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115480851507-30472-0/krb5kdc/oryx_127.29.194.1.keytab
--principal=oryx/127.29.194.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.194.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115480851507-30472-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.194.1
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.29.194.62:34237
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9122
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.194.1
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115480851507-30472-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:11:57.982389 9122 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:11:57.982646 9122 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:11:57.988776 9127 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:11:57.988798 9128 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:11:57.988922 9122 server_base.cc:1034] running on GCE node
W20260807 15:11:57.988822 9130 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:11:57.989279 9122 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:11:57.989581 9122 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:11:57.990760 9122 hybrid_clock.cc:648] HybridClock initialized: now 1786115517990743 us; error 33 us; skew 500 ppm
Aug 07 15:11:57 dist-test-slave-42z9 krb5kdc[8938](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115517, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.194.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:11:57.993920 9122 init.cc:377] Logged in from keytab as oryx/127.29.194.1@KRBTEST.COM (short username oryx)
I20260807 15:11:57.994808 9122 webserver.cc:460] Webserver started at http://127.29.194.1:34045/ using document root <none> and password file <none>
I20260807 15:11:57.995108 9122 fs_manager.cc:359] Metadata directory not provided
I20260807 15:11:57.995169 9122 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:11:57.995328 9122 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:11:57.996771 9122 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115480851507-30472-0/minicluster-data/ts-0/data/instance:
uuid: "1c708c44654e461099098050b30d16f0"
format_stamp: "Formatted at 2026-08-07 15:11:57 on dist-test-slave-42z9"
server_key: "f41e6e701e2b656cd0946ac95e48c581"
server_key_iv: "6cafe84acaf860f41ce8ba5c154cb935"
server_key_version: "encryptionkey@0"
I20260807 15:11:57.997188 9122 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115480851507-30472-0/minicluster-data/ts-0/wal/instance:
uuid: "1c708c44654e461099098050b30d16f0"
format_stamp: "Formatted at 2026-08-07 15:11:57 on dist-test-slave-42z9"
server_key: "f41e6e701e2b656cd0946ac95e48c581"
server_key_iv: "6cafe84acaf860f41ce8ba5c154cb935"
server_key_version: "encryptionkey@0"
I20260807 15:11:57.998987 9122 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.004s sys 0.000s
I20260807 15:11:58.000113 9142 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:11:58.000517 9122 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:11:58.000737 9122 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115480851507-30472-0/minicluster-data/ts-0/data,/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115480851507-30472-0/minicluster-data/ts-0/wal
uuid: "1c708c44654e461099098050b30d16f0"
format_stamp: "Formatted at 2026-08-07 15:11:57 on dist-test-slave-42z9"
server_key: "f41e6e701e2b656cd0946ac95e48c581"
server_key_iv: "6cafe84acaf860f41ce8ba5c154cb935"
server_key_version: "encryptionkey@0"
I20260807 15:11:58.000844 9122 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115480851507-30472-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115480851507-30472-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115480851507-30472-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:11:58.021253 9122 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:11:58.024335 9122 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:11:58.024555 9122 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:11:58.025144 9122 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:11:58.025192 9122 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:11:58.025224 9122 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:11:58.025241 9122 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 15:11:58 dist-test-slave-42z9 krb5kdc[8938](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115517, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.194.1@KRBTEST.COM for oryx/127.29.194.62@KRBTEST.COM
I20260807 15:11:58.032624 9122 rpc_server.cc:307] RPC server started. Bound to: 127.29.194.1:36541
I20260807 15:11:58.032727 9302 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.194.1:36541 every 8 connection(s)
I20260807 15:11:58.033072 9122 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115480851507-30472-0/minicluster-data/ts-0/data/info.pb
I20260807 15:11:58.035404 9088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:58.025756 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.194.1:54843 (local address 127.29.194.62:34237)
0807 15:11:58.025827 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:58.025829 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:58.026112 (+ 283us) server_negotiation.cc:407] Connection header received
0807 15:11:58.026790 (+ 678us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:58.026801 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:58.026822 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:58.026935 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:11:58.027991 (+ 1056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:58.028545 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:58.029714 (+ 1169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:58.029933 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:58.029994 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:58.032411 (+ 2417us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:11:58.032413 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:11:58.032414 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:11:58.032424 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:11:58.034060 (+ 1636us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:58.034553 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:58.034554 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:58.034554 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:58.034621 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:58.034831 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:58.034831 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:58.034832 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:58.034952 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:11:58.035000 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0807 15:11:58.035237 (+ 237us) server_negotiation.cc:299] Negotiation successful
0807 15:11:58.035332 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260807 15:11:58.035624 9158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:58.025718 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:34237 (local address 127.29.194.1:54843)
0807 15:11:58.025893 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0807 15:11:58.025941 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0807 15:11:58.026657 (+ 716us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:11:58.027068 (+ 411us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:11:58.027074 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:11:58.027524 (+ 450us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:11:58.027858 (+ 334us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:58.027864 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:58.028660 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:58.028661 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:58.029618 (+ 957us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:58.029623 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:58.030026 (+ 403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:58.030027 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:58.030195 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:58.030760 (+ 565us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:11:58.030768 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:11:58.032298 (+ 1530us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:11:58.034175 (+ 1877us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:11:58.034183 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:11:58.034185 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:11:58.034448 (+ 263us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:11:58.034680 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:11:58.034681 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:11:58.034681 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:11:58.034784 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:11:58.034995 (+ 211us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:11:58.034999 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:11:58.035135 (+ 136us) client_negotiation.cc:769] Sending connection context
0807 15:11:58.035222 (+ 87us) client_negotiation.cc:240] Negotiation successful
0807 15:11:58.035366 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":85,"threads_started":1}
I20260807 15:11:58.036149 9304 heartbeater.cc:347] Connected to a master server at 127.29.194.62:34237
I20260807 15:11:58.036242 9304 heartbeater.cc:464] Registering TS with master...
I20260807 15:11:58.036630 9304 heartbeater.cc:511] Master 127.29.194.62:34237 requested a full tablet report, sending...
I20260807 15:11:58.037545 9008 ts_manager.cc:194] Registered new tserver with Master: 1c708c44654e461099098050b30d16f0 (127.29.194.1:36541)
I20260807 15:11:58.038476 9008 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.29.194.1@KRBTEST.COM'} at 127.29.194.1:54843
I20260807 15:11:58.039448 30472 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7fZSIL/build/release/bin/kudu as pid 9122
I20260807 15:11:58.039539 30472 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115480851507-30472-0/minicluster-data/ts-0/wal/instance
I20260807 15:11:58.039740 30472 external_mini_cluster.cc:1428] Setting key de34445a34014f46fabe40e37462efab
I20260807 15:11:58.045235 9312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:58.037793 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:34237 (local address 127.29.194.1:43995)
0807 15:11:58.038035 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0807 15:11:58.038049 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 15:11:58.038130 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:11:58.038258 (+ 128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:11:58.038259 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:11:58.038478 (+ 219us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:11:58.038591 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:58.038595 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:58.039658 (+ 1063us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:58.039660 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:58.040357 (+ 697us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:58.040360 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:58.040857 (+ 497us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:58.040858 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:58.040941 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:58.041577 (+ 636us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:11:58.041582 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:11:58.042008 (+ 426us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:11:58.044080 (+ 2072us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:11:58.044081 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:11:58.044081 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:11:58.044301 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:11:58.044541 (+ 240us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:11:58.044542 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:11:58.044542 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:11:58.044607 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:11:58.044905 (+ 298us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:11:58.044906 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:11:58.044976 (+ 70us) client_negotiation.cc:769] Sending connection context
0807 15:11:58.045036 (+ 60us) client_negotiation.cc:240] Negotiation successful
0807 15:11:58.045147 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":104,"threads_started":1}
I20260807 15:11:58.045292 9088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:58.037874 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.194.1:43995 (local address 127.29.194.62:34237)
0807 15:11:58.037943 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:58.037946 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:58.038156 (+ 210us) server_negotiation.cc:407] Connection header received
0807 15:11:58.038182 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:58.038183 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:58.038194 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:58.038245 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:11:58.038667 (+ 422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:58.039462 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:58.040492 (+ 1030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:58.040779 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:58.040851 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:58.042134 (+ 1283us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:11:58.042136 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:11:58.042136 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:11:58.042144 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:11:58.043960 (+ 1816us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:58.044386 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:58.044387 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:58.044388 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:58.044467 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:58.044667 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:58.044668 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:58.044669 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:58.044827 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:11:58.044898 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0807 15:11:58.045079 (+ 181us) server_negotiation.cc:299] Negotiation successful
0807 15:11:58.045181 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
WARNING: no policy specified for oryx/127.29.194.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.29.194.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.29.194.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115480851507-30472-0/krb5kdc/oryx_127.29.194.2.keytab.
Entry for principal oryx/127.29.194.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115480851507-30472-0/krb5kdc/oryx_127.29.194.2.keytab.
I20260807 15:11:58.073079 30472 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7fZSIL/build/release/bin/kudu
/tmp/dist-test-task7fZSIL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115480851507-30472-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115480851507-30472-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115480851507-30472-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115480851507-30472-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.194.2:0
--local_ip_for_outbound_sockets=127.29.194.2
--webserver_interface=127.29.194.2
--webserver_port=0
--tserver_master_addrs=127.29.194.62:34237
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115480851507-30472-0/krb5kdc/oryx_127.29.194.2.keytab
--principal=oryx/127.29.194.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.194.20:44921
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115480851507-30472-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115480851507-30472-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115480851507-30472-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:11:58.163480 9318 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:11:58.163722 9318 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:11:58.163748 9318 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:11:58.165051 9318 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:11:58.165117 9318 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:11:58.165139 9318 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 15:11:58.165158 9318 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.194.2
I20260807 15:11:58.166973 9318 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.194.20:44921
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115480851507-30472-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115480851507-30472-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115480851507-30472-0/krb5kdc/oryx_127.29.194.2.keytab
--principal=oryx/127.29.194.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.194.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115480851507-30472-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.194.2
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.29.194.62:34237
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9318
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.194.2
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115480851507-30472-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:11:58.167254 9318 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:11:58.167522 9318 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:11:58.174127 9327 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:11:58.174151 9329 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:11:58.174355 9318 server_base.cc:1034] running on GCE node
W20260807 15:11:58.174151 9331 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:11:58.174762 9318 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:11:58.175050 9318 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:11:58.176229 9318 hybrid_clock.cc:648] HybridClock initialized: now 1786115518176210 us; error 44 us; skew 500 ppm
Aug 07 15:11:58 dist-test-slave-42z9 krb5kdc[8938](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115518, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.194.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:11:58.179436 9318 init.cc:377] Logged in from keytab as oryx/127.29.194.2@KRBTEST.COM (short username oryx)
I20260807 15:11:58.180258 9318 webserver.cc:460] Webserver started at http://127.29.194.2:36367/ using document root <none> and password file <none>
I20260807 15:11:58.180631 9318 fs_manager.cc:359] Metadata directory not provided
I20260807 15:11:58.180693 9318 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:11:58.180903 9318 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:11:58.182564 9318 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115480851507-30472-0/minicluster-data/ts-1/data/instance:
uuid: "7c8b57fd081d41bb8f74cf365d3e0a7a"
format_stamp: "Formatted at 2026-08-07 15:11:58 on dist-test-slave-42z9"
server_key: "a0dc9f6d87531fc80752a611921f4a28"
server_key_iv: "13f1b3c48869f4794d7b41ddb3a251e9"
server_key_version: "encryptionkey@0"
I20260807 15:11:58.183136 9318 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115480851507-30472-0/minicluster-data/ts-1/wal/instance:
uuid: "7c8b57fd081d41bb8f74cf365d3e0a7a"
format_stamp: "Formatted at 2026-08-07 15:11:58 on dist-test-slave-42z9"
server_key: "a0dc9f6d87531fc80752a611921f4a28"
server_key_iv: "13f1b3c48869f4794d7b41ddb3a251e9"
server_key_version: "encryptionkey@0"
I20260807 15:11:58.184968 9318 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20260807 15:11:58.186167 9343 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:11:58.186506 9318 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260807 15:11:58.186653 9318 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115480851507-30472-0/minicluster-data/ts-1/data,/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115480851507-30472-0/minicluster-data/ts-1/wal
uuid: "7c8b57fd081d41bb8f74cf365d3e0a7a"
format_stamp: "Formatted at 2026-08-07 15:11:58 on dist-test-slave-42z9"
server_key: "a0dc9f6d87531fc80752a611921f4a28"
server_key_iv: "13f1b3c48869f4794d7b41ddb3a251e9"
server_key_version: "encryptionkey@0"
I20260807 15:11:58.186760 9318 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115480851507-30472-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115480851507-30472-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115480851507-30472-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:11:58.197952 9318 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:11:58.200204 9318 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:11:58.200367 9318 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:11:58.200940 9318 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:11:58.200989 9318 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:11:58.201020 9318 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:11:58.201047 9318 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:11:58.207696 9318 rpc_server.cc:307] RPC server started. Bound to: 127.29.194.2:36405
I20260807 15:11:58.207762 9505 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.194.2:36405 every 8 connection(s)
Aug 07 15:11:58 dist-test-slave-42z9 krb5kdc[8938](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115518, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.194.2@KRBTEST.COM for oryx/127.29.194.62@KRBTEST.COM
I20260807 15:11:58.208163 9318 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115480851507-30472-0/minicluster-data/ts-1/data/info.pb
I20260807 15:11:58.209903 30472 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7fZSIL/build/release/bin/kudu as pid 9318
I20260807 15:11:58.210037 30472 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115480851507-30472-0/minicluster-data/ts-1/wal/instance
I20260807 15:11:58.210258 30472 external_mini_cluster.cc:1428] Setting key 8af6b547ad7935e22d788c3bb8356002
I20260807 15:11:58.212283 9088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:58.201480 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.194.2:37209 (local address 127.29.194.62:34237)
0807 15:11:58.201540 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:58.201543 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:58.201722 (+ 179us) server_negotiation.cc:407] Connection header received
0807 15:11:58.202296 (+ 574us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:58.202297 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:58.202310 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:58.202361 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:11:58.203243 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:58.204039 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:58.205469 (+ 1430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:58.205766 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:58.205835 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:58.208557 (+ 2722us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:11:58.208559 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:11:58.208559 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:11:58.208569 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:11:58.210699 (+ 2130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:58.211181 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:58.211182 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:58.211182 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:58.211243 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:58.211528 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:58.211528 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:58.211528 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:58.211677 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:11:58.211738 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0807 15:11:58.212119 (+ 381us) server_negotiation.cc:299] Negotiation successful
0807 15:11:58.212202 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260807 15:11:58.212579 9353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:58.201397 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:34237 (local address 127.29.194.2:37209)
0807 15:11:58.201593 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0807 15:11:58.201635 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0807 15:11:58.202183 (+ 548us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:11:58.202463 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:11:58.202468 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:11:58.202850 (+ 382us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:11:58.203159 (+ 309us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:58.203166 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:58.204137 (+ 971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:58.204138 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:58.205352 (+ 1214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:58.205356 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:58.205849 (+ 493us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:58.205850 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:58.206010 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:58.206567 (+ 557us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:11:58.206580 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:11:58.208447 (+ 1867us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:11:58.210787 (+ 2340us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:11:58.210796 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:11:58.210798 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:11:58.211106 (+ 308us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:11:58.211295 (+ 189us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:11:58.211296 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:11:58.211297 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:11:58.211463 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:11:58.211742 (+ 279us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:11:58.211750 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:11:58.211937 (+ 187us) client_negotiation.cc:769] Sending connection context
0807 15:11:58.212065 (+ 128us) client_negotiation.cc:240] Negotiation successful
0807 15:11:58.212249 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":124,"threads_started":1}
I20260807 15:11:58.213665 9506 heartbeater.cc:347] Connected to a master server at 127.29.194.62:34237
I20260807 15:11:58.213876 9506 heartbeater.cc:464] Registering TS with master...
I20260807 15:11:58.214188 9506 heartbeater.cc:511] Master 127.29.194.62:34237 requested a full tablet report, sending...
I20260807 15:11:58.214859 9008 ts_manager.cc:194] Registered new tserver with Master: 7c8b57fd081d41bb8f74cf365d3e0a7a (127.29.194.2:36405)
I20260807 15:11:58.215411 9008 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.29.194.2@KRBTEST.COM'} at 127.29.194.2:37209
I20260807 15:11:58.221232 9088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:58.214870 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.194.2:59609 (local address 127.29.194.62:34237)
0807 15:11:58.214930 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:58.214932 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:58.215002 (+ 70us) server_negotiation.cc:407] Connection header received
0807 15:11:58.215095 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:58.215096 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:58.215109 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:58.215155 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:11:58.215523 (+ 368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:58.216124 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:58.216891 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:58.217102 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:58.217160 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:58.218448 (+ 1288us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:11:58.218450 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:11:58.218450 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:11:58.218458 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:11:58.219997 (+ 1539us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:58.220388 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:58.220389 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:58.220390 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:58.220447 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:58.220628 (+ 181us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:58.220628 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:58.220629 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:58.220799 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:11:58.220863 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0807 15:11:58.221023 (+ 160us) server_negotiation.cc:299] Negotiation successful
0807 15:11:58.221127 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260807 15:11:58.221235 9518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:58.214797 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:34237 (local address 127.29.194.2:59609)
0807 15:11:58.214931 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0807 15:11:58.214942 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 15:11:58.215031 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:11:58.215159 (+ 128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:11:58.215159 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:11:58.215352 (+ 193us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:11:58.215414 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:58.215416 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:58.216221 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:58.216223 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:58.216812 (+ 589us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:58.216814 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:58.217167 (+ 353us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:58.217168 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:58.217241 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:58.217965 (+ 724us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:11:58.217971 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:11:58.218310 (+ 339us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:11:58.220109 (+ 1799us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:11:58.220110 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:11:58.220111 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:11:58.220288 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:11:58.220511 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:11:58.220511 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:11:58.220512 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:11:58.220572 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:11:58.220872 (+ 300us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:11:58.220873 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:11:58.220928 (+ 55us) client_negotiation.cc:769] Sending connection context
0807 15:11:58.220985 (+ 57us) client_negotiation.cc:240] Negotiation successful
0807 15:11:58.221125 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":81,"threads_started":1}
WARNING: no policy specified for oryx/127.29.194.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.29.194.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.29.194.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115480851507-30472-0/krb5kdc/oryx_127.29.194.3.keytab.
Entry for principal oryx/127.29.194.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115480851507-30472-0/krb5kdc/oryx_127.29.194.3.keytab.
I20260807 15:11:58.245689 30472 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7fZSIL/build/release/bin/kudu
/tmp/dist-test-task7fZSIL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115480851507-30472-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115480851507-30472-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115480851507-30472-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115480851507-30472-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.194.3:0
--local_ip_for_outbound_sockets=127.29.194.3
--webserver_interface=127.29.194.3
--webserver_port=0
--tserver_master_addrs=127.29.194.62:34237
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115480851507-30472-0/krb5kdc/oryx_127.29.194.3.keytab
--principal=oryx/127.29.194.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.194.20:44921
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115480851507-30472-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115480851507-30472-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115480851507-30472-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:11:58.336758 9522 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:11:58.337023 9522 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:11:58.337065 9522 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:11:58.338419 9522 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:11:58.338481 9522 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:11:58.338511 9522 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 15:11:58.338546 9522 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.194.3
I20260807 15:11:58.340314 9522 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.194.20:44921
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115480851507-30472-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115480851507-30472-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115480851507-30472-0/krb5kdc/oryx_127.29.194.3.keytab
--principal=oryx/127.29.194.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.194.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115480851507-30472-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.194.3
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.29.194.62:34237
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9522
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.194.3
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115480851507-30472-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:11:58.340612 9522 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:11:58.340862 9522 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:11:58.346563 9534 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:11:58.346532 9533 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:11:58.346586 9522 server_base.cc:1034] running on GCE node
W20260807 15:11:58.346532 9536 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:11:58.347074 9522 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:11:58.347354 9522 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:11:58.348536 9522 hybrid_clock.cc:648] HybridClock initialized: now 1786115518348520 us; error 39 us; skew 500 ppm
Aug 07 15:11:58 dist-test-slave-42z9 krb5kdc[8938](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115518, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.194.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:11:58.351459 9522 init.cc:377] Logged in from keytab as oryx/127.29.194.3@KRBTEST.COM (short username oryx)
I20260807 15:11:58.351981 9522 webserver.cc:460] Webserver started at http://127.29.194.3:38343/ using document root <none> and password file <none>
I20260807 15:11:58.352227 9522 fs_manager.cc:359] Metadata directory not provided
I20260807 15:11:58.352278 9522 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:11:58.352413 9522 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:11:58.353559 9522 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115480851507-30472-0/minicluster-data/ts-2/data/instance:
uuid: "86b01224f5b34c19a27d23a31ab7306a"
format_stamp: "Formatted at 2026-08-07 15:11:58 on dist-test-slave-42z9"
server_key: "4bc12a958027492c4636205115d4285f"
server_key_iv: "a46f52aa48b6675c6e690a373b6f95f8"
server_key_version: "encryptionkey@0"
I20260807 15:11:58.353910 9522 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115480851507-30472-0/minicluster-data/ts-2/wal/instance:
uuid: "86b01224f5b34c19a27d23a31ab7306a"
format_stamp: "Formatted at 2026-08-07 15:11:58 on dist-test-slave-42z9"
server_key: "4bc12a958027492c4636205115d4285f"
server_key_iv: "a46f52aa48b6675c6e690a373b6f95f8"
server_key_version: "encryptionkey@0"
I20260807 15:11:58.355289 9522 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 15:11:58.356153 9545 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:11:58.356361 9522 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:11:58.356465 9522 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115480851507-30472-0/minicluster-data/ts-2/data,/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115480851507-30472-0/minicluster-data/ts-2/wal
uuid: "86b01224f5b34c19a27d23a31ab7306a"
format_stamp: "Formatted at 2026-08-07 15:11:58 on dist-test-slave-42z9"
server_key: "4bc12a958027492c4636205115d4285f"
server_key_iv: "a46f52aa48b6675c6e690a373b6f95f8"
server_key_version: "encryptionkey@0"
I20260807 15:11:58.356529 9522 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115480851507-30472-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115480851507-30472-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115480851507-30472-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:11:58.376394 9522 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:11:58.378692 9522 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:11:58.378850 9522 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:11:58.379413 9522 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:11:58.379460 9522 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:11:58.379498 9522 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:11:58.379526 9522 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 15:11:58 dist-test-slave-42z9 krb5kdc[8938](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115518, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.194.3@KRBTEST.COM for oryx/127.29.194.62@KRBTEST.COM
I20260807 15:11:58.387526 9522 rpc_server.cc:307] RPC server started. Bound to: 127.29.194.3:44857
I20260807 15:11:58.387634 9707 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.194.3:44857 every 8 connection(s)
I20260807 15:11:58.388110 9522 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115480851507-30472-0/minicluster-data/ts-2/data/info.pb
I20260807 15:11:58.391047 9088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:58.380019 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.194.3:33297 (local address 127.29.194.62:34237)
0807 15:11:58.380074 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:58.380077 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:58.380326 (+ 249us) server_negotiation.cc:407] Connection header received
0807 15:11:58.380946 (+ 620us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:58.380947 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:58.380959 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:58.381020 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:11:58.382017 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:58.382813 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:58.384404 (+ 1591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:58.384658 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:58.384748 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:58.388036 (+ 3288us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:11:58.388037 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:11:58.388038 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:11:58.388047 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:11:58.389625 (+ 1578us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:58.390133 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:58.390134 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:58.390134 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:58.390192 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:58.390427 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:58.390429 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:58.390429 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:58.390568 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:11:58.390636 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0807 15:11:58.390869 (+ 233us) server_negotiation.cc:299] Negotiation successful
0807 15:11:58.390955 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260807 15:11:58.391265 9561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:58.380041 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:34237 (local address 127.29.194.3:33297)
0807 15:11:58.380201 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0807 15:11:58.380238 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0807 15:11:58.380826 (+ 588us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:11:58.381142 (+ 316us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:11:58.381147 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:11:58.381578 (+ 431us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:11:58.381901 (+ 323us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:58.381909 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:58.382921 (+ 1012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:58.382922 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:58.384296 (+ 1374us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:58.384301 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:58.384755 (+ 454us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:58.384757 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:58.384916 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:58.385610 (+ 694us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:11:58.385620 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:11:58.387921 (+ 2301us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:11:58.389729 (+ 1808us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:11:58.389738 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:11:58.389740 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:11:58.390062 (+ 322us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:11:58.390242 (+ 180us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:11:58.390242 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:11:58.390243 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:11:58.390345 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:11:58.390631 (+ 286us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:11:58.390636 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:11:58.390768 (+ 132us) client_negotiation.cc:769] Sending connection context
0807 15:11:58.390853 (+ 85us) client_negotiation.cc:240] Negotiation successful
0807 15:11:58.391032 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":81,"threads_started":1}
I20260807 15:11:58.391789 9708 heartbeater.cc:347] Connected to a master server at 127.29.194.62:34237
I20260807 15:11:58.391893 9708 heartbeater.cc:464] Registering TS with master...
I20260807 15:11:58.392112 9708 heartbeater.cc:511] Master 127.29.194.62:34237 requested a full tablet report, sending...
I20260807 15:11:58.392555 9009 ts_manager.cc:194] Registered new tserver with Master: 86b01224f5b34c19a27d23a31ab7306a (127.29.194.3:44857)
I20260807 15:11:58.393062 9009 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.29.194.3@KRBTEST.COM'} at 127.29.194.3:33297
I20260807 15:11:58.394115 30472 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7fZSIL/build/release/bin/kudu as pid 9522
I20260807 15:11:58.394251 30472 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115480851507-30472-0/minicluster-data/ts-2/wal/instance
I20260807 15:11:58.394477 30472 external_mini_cluster.cc:1428] Setting key 61eb00bfaa0d63066c1c0a7b3ffe0275
I20260807 15:11:58.395387 30472 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 15:11:58.399325 9088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:58.392927 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.194.3:53815 (local address 127.29.194.62:34237)
0807 15:11:58.392971 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:58.392973 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:58.393283 (+ 310us) server_negotiation.cc:407] Connection header received
0807 15:11:58.393333 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:58.393334 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:58.393345 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:58.393400 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:11:58.393903 (+ 503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:58.394778 (+ 875us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:58.395614 (+ 836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:58.395812 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:58.395859 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:58.396919 (+ 1060us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:11:58.396920 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:11:58.396920 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:11:58.396927 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:11:58.398282 (+ 1355us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:58.398661 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:58.398661 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:58.398662 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:58.398713 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:58.398902 (+ 189us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:58.398902 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:58.398903 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:58.399016 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:11:58.399064 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0807 15:11:58.399197 (+ 133us) server_negotiation.cc:299] Negotiation successful
0807 15:11:58.399261 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260807 15:11:58.399359 9720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:58.392871 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:34237 (local address 127.29.194.3:53815)
0807 15:11:58.393226 (+ 355us) negotiation.cc:107] Waiting for socket to connect
0807 15:11:58.393236 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 15:11:58.393294 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:11:58.393418 (+ 124us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:11:58.393419 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:11:58.393672 (+ 253us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:11:58.393738 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:58.393740 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:58.394920 (+ 1180us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:58.394922 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:58.395535 (+ 613us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:58.395538 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:58.395873 (+ 335us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:58.395874 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:58.395941 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:58.396408 (+ 467us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:11:58.396413 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:11:58.396805 (+ 392us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:11:58.398366 (+ 1561us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:11:58.398367 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:11:58.398368 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:11:58.398593 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:11:58.398766 (+ 173us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:11:58.398766 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:11:58.398767 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:11:58.398833 (+ 66us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:11:58.399072 (+ 239us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:11:58.399073 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:11:58.399119 (+ 46us) client_negotiation.cc:769] Sending connection context
0807 15:11:58.399175 (+ 56us) client_negotiation.cc:240] Negotiation successful
0807 15:11:58.399265 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":328,"thread_start_us":86,"threads_started":1}
Aug 07 15:11:58 dist-test-slave-42z9 krb5kdc[8938](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115518, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 07 15:11:58 dist-test-slave-42z9 krb5kdc[8938](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.29.194.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 15:11:58 dist-test-slave-42z9 krb5kdc[8938](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.29.194.62@KRBTEST.COM, Server not found in Kerberos database
W20260807 15:11:58.421576 9088 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:11:58.416171 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56724 (local address 127.29.194.62:34237)
0807 15:11:58.416241 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:58.416243 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:58.416252 (+ 9us) server_negotiation.cc:407] Connection header received
0807 15:11:58.416276 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:58.416277 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:58.416286 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:58.416338 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:11:58.416940 (+ 602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:58.417489 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:58.418224 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:58.418410 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:58.418466 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:58.421508 (+ 3042us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56724: BlockingRecv error: recv got EOF from 127.0.0.1:56724 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
Aug 07 15:11:58 dist-test-slave-42z9 krb5kdc[8938](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115518, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.29.194.62@KRBTEST.COM
I20260807 15:11:58.430637 9088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:58.422608 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56732 (local address 127.29.194.62:34237)
0807 15:11:58.422675 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:58.422677 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:58.422723 (+ 46us) server_negotiation.cc:407] Connection header received
0807 15:11:58.422826 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:58.422828 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:58.422846 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:58.422915 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:11:58.423635 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:58.424124 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:58.424939 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:58.425137 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:58.425201 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:58.427815 (+ 2614us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:11:58.427817 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:11:58.427817 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:11:58.427825 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:11:58.429151 (+ 1326us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:58.429647 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:58.429648 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:58.429648 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:58.429710 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:58.429955 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:58.429955 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:58.429956 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:58.430067 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:11:58.430146 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0807 15:11:58.430411 (+ 265us) server_negotiation.cc:299] Negotiation successful
0807 15:11:58.430542 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260807 15:11:58.432436 9009 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56732:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 15:11:58.434201 9009 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 15:11:58.441279 9747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:58.436913 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.3:44857 (local address 127.0.0.1:46368)
0807 15:11:58.437340 (+ 427us) negotiation.cc:107] Waiting for socket to connect
0807 15:11:58.437347 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0807 15:11:58.437388 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:11:58.437676 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:11:58.437677 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:11:58.437679 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:11:58.437736 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:58.437737 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:58.438599 (+ 862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:58.438600 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:58.439708 (+ 1108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:58.439711 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:58.440915 (+ 1204us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:58.440916 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:58.441012 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:58.441032 (+ 20us) client_negotiation.cc:769] Sending connection context
0807 15:11:58.441111 (+ 79us) client_negotiation.cc:240] Negotiation successful
0807 15:11:58.441181 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":409,"thread_start_us":92,"threads_started":1}
I20260807 15:11:58.441556 9745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:58.436830 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34040 (local address 127.29.194.1:36541)
0807 15:11:58.437032 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:58.437037 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:58.437267 (+ 230us) server_negotiation.cc:407] Connection header received
0807 15:11:58.437322 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:58.437325 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:58.437342 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:58.437682 (+ 340us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:11:58.437740 (+ 58us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:58.438997 (+ 1257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:58.440258 (+ 1261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:58.441008 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:58.441101 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:58.441192 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0807 15:11:58.441301 (+ 109us) server_negotiation.cc:299] Negotiation successful
0807 15:11:58.441352 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":166,"thread_start_us":92,"threads_started":1}
I20260807 15:11:58.441830 9748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:58.437034 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46368 (local address 127.29.194.3:44857)
0807 15:11:58.437183 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:58.437187 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:58.437477 (+ 290us) server_negotiation.cc:407] Connection header received
0807 15:11:58.437525 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:58.437527 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:58.437543 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:58.437610 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:11:58.437812 (+ 202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:58.438512 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:58.439897 (+ 1385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:58.440794 (+ 897us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:58.440904 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:58.441013 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0807 15:11:58.441571 (+ 558us) server_negotiation.cc:299] Negotiation successful
0807 15:11:58.441652 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":116,"thread_start_us":85,"threads_started":1}
I20260807 15:11:58.441933 9741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:58.436704 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.1:36541 (local address 127.0.0.1:34040)
0807 15:11:58.437188 (+ 484us) negotiation.cc:107] Waiting for socket to connect
0807 15:11:58.437197 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0807 15:11:58.437249 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:11:58.437507 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:11:58.437511 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:11:58.437514 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:11:58.437601 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:58.437604 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:58.439183 (+ 1579us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:58.439184 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:58.440156 (+ 972us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:58.440159 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:58.441103 (+ 944us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:58.441104 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:58.441203 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:58.441212 (+ 9us) client_negotiation.cc:769] Sending connection context
0807 15:11:58.441280 (+ 68us) client_negotiation.cc:240] Negotiation successful
0807 15:11:58.441352 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":435,"mutex_wait_us":17,"thread_start_us":72,"threads_started":1}
I20260807 15:11:58.442356 9740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:58.436704 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.2:36405 (local address 127.0.0.1:35692)
0807 15:11:58.437028 (+ 324us) negotiation.cc:107] Waiting for socket to connect
0807 15:11:58.437101 (+ 73us) client_negotiation.cc:174] Beginning negotiation
0807 15:11:58.437237 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:11:58.437614 (+ 377us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:11:58.437615 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:11:58.437617 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:11:58.437658 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:58.437659 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:58.440245 (+ 2586us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:58.440245 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:58.440935 (+ 690us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:58.440936 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:58.442088 (+ 1152us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:58.442089 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:58.442167 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:58.442173 (+ 6us) client_negotiation.cc:769] Sending connection context
0807 15:11:58.442226 (+ 53us) client_negotiation.cc:240] Negotiation successful
0807 15:11:58.442284 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":296,"thread_start_us":57,"threads_started":1}
I20260807 15:11:58.442651 9742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:58.436822 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35692 (local address 127.29.194.2:36405)
0807 15:11:58.437108 (+ 286us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:58.437112 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:58.437193 (+ 81us) server_negotiation.cc:407] Connection header received
0807 15:11:58.437414 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:58.437417 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:58.437433 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:58.437521 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:11:58.437736 (+ 215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:58.438997 (+ 1261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:58.441061 (+ 2064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:58.441872 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:58.441958 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:58.442044 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0807 15:11:58.442495 (+ 451us) server_negotiation.cc:299] Negotiation successful
0807 15:11:58.442544 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":256,"thread_start_us":103,"threads_started":1}
I20260807 15:11:58.442744 9612 tablet_service.cc:1467] Processing CreateTablet for tablet f79d6eba55f347adbd2e7544f0aee703 (DEFAULT_TABLE table=test-table [id=d7dd955011f64e49a98093a94cbb58b5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:11:58.442967 9214 tablet_service.cc:1467] Processing CreateTablet for tablet f79d6eba55f347adbd2e7544f0aee703 (DEFAULT_TABLE table=test-table [id=d7dd955011f64e49a98093a94cbb58b5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:11:58.443173 9612 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f79d6eba55f347adbd2e7544f0aee703. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:11:58.443472 9214 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f79d6eba55f347adbd2e7544f0aee703. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:11:58.446458 9749 tablet_bootstrap.cc:492] T f79d6eba55f347adbd2e7544f0aee703 P 86b01224f5b34c19a27d23a31ab7306a: Bootstrap starting.
I20260807 15:11:58.447165 9749 tablet_bootstrap.cc:654] T f79d6eba55f347adbd2e7544f0aee703 P 86b01224f5b34c19a27d23a31ab7306a: Neither blocks nor log segments found. Creating new log.
I20260807 15:11:58.447513 9750 tablet_bootstrap.cc:492] T f79d6eba55f347adbd2e7544f0aee703 P 1c708c44654e461099098050b30d16f0: Bootstrap starting.
I20260807 15:11:58.447532 9749 log.cc:824] T f79d6eba55f347adbd2e7544f0aee703 P 86b01224f5b34c19a27d23a31ab7306a: Log is configured to *not* fsync() on all Append() calls
I20260807 15:11:58.448293 9749 tablet_bootstrap.cc:492] T f79d6eba55f347adbd2e7544f0aee703 P 86b01224f5b34c19a27d23a31ab7306a: No bootstrap required, opened a new log
I20260807 15:11:58.448341 9750 tablet_bootstrap.cc:654] T f79d6eba55f347adbd2e7544f0aee703 P 1c708c44654e461099098050b30d16f0: Neither blocks nor log segments found. Creating new log.
I20260807 15:11:58.448365 9749 ts_tablet_manager.cc:1397] T f79d6eba55f347adbd2e7544f0aee703 P 86b01224f5b34c19a27d23a31ab7306a: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 15:11:58.448304 9410 tablet_service.cc:1467] Processing CreateTablet for tablet f79d6eba55f347adbd2e7544f0aee703 (DEFAULT_TABLE table=test-table [id=d7dd955011f64e49a98093a94cbb58b5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:11:58.448655 9750 log.cc:824] T f79d6eba55f347adbd2e7544f0aee703 P 1c708c44654e461099098050b30d16f0: Log is configured to *not* fsync() on all Append() calls
I20260807 15:11:58.448734 9410 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f79d6eba55f347adbd2e7544f0aee703. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:11:58.449704 9750 tablet_bootstrap.cc:492] T f79d6eba55f347adbd2e7544f0aee703 P 1c708c44654e461099098050b30d16f0: No bootstrap required, opened a new log
I20260807 15:11:58.449786 9750 ts_tablet_manager.cc:1397] T f79d6eba55f347adbd2e7544f0aee703 P 1c708c44654e461099098050b30d16f0: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260807 15:11:58.450274 9749 raft_consensus.cc:348] T f79d6eba55f347adbd2e7544f0aee703 P 86b01224f5b34c19a27d23a31ab7306a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c708c44654e461099098050b30d16f0" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 36541 } } peers { permanent_uuid: "86b01224f5b34c19a27d23a31ab7306a" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 44857 } } peers { permanent_uuid: "7c8b57fd081d41bb8f74cf365d3e0a7a" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 36405 } }
I20260807 15:11:58.450433 9749 raft_consensus.cc:374] T f79d6eba55f347adbd2e7544f0aee703 P 86b01224f5b34c19a27d23a31ab7306a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:11:58.450465 9749 raft_consensus.cc:729] T f79d6eba55f347adbd2e7544f0aee703 P 86b01224f5b34c19a27d23a31ab7306a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 86b01224f5b34c19a27d23a31ab7306a, State: Initialized, Role: FOLLOWER
I20260807 15:11:58.450614 9749 consensus_queue.cc:260] T f79d6eba55f347adbd2e7544f0aee703 P 86b01224f5b34c19a27d23a31ab7306a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c708c44654e461099098050b30d16f0" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 36541 } } peers { permanent_uuid: "86b01224f5b34c19a27d23a31ab7306a" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 44857 } } peers { permanent_uuid: "7c8b57fd081d41bb8f74cf365d3e0a7a" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 36405 } }
I20260807 15:11:58.450932 9749 ts_tablet_manager.cc:1428] T f79d6eba55f347adbd2e7544f0aee703 P 86b01224f5b34c19a27d23a31ab7306a: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260807 15:11:58.451010 9708 heartbeater.cc:503] Master 127.29.194.62:34237 was elected leader, sending a full tablet report...
I20260807 15:11:58.452461 9750 raft_consensus.cc:348] T f79d6eba55f347adbd2e7544f0aee703 P 1c708c44654e461099098050b30d16f0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c708c44654e461099098050b30d16f0" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 36541 } } peers { permanent_uuid: "86b01224f5b34c19a27d23a31ab7306a" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 44857 } } peers { permanent_uuid: "7c8b57fd081d41bb8f74cf365d3e0a7a" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 36405 } }
I20260807 15:11:58.452661 9750 raft_consensus.cc:374] T f79d6eba55f347adbd2e7544f0aee703 P 1c708c44654e461099098050b30d16f0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:11:58.452932 9750 raft_consensus.cc:729] T f79d6eba55f347adbd2e7544f0aee703 P 1c708c44654e461099098050b30d16f0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1c708c44654e461099098050b30d16f0, State: Initialized, Role: FOLLOWER
I20260807 15:11:58.453151 9750 consensus_queue.cc:260] T f79d6eba55f347adbd2e7544f0aee703 P 1c708c44654e461099098050b30d16f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c708c44654e461099098050b30d16f0" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 36541 } } peers { permanent_uuid: "86b01224f5b34c19a27d23a31ab7306a" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 44857 } } peers { permanent_uuid: "7c8b57fd081d41bb8f74cf365d3e0a7a" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 36405 } }
I20260807 15:11:58.453684 9750 ts_tablet_manager.cc:1428] T f79d6eba55f347adbd2e7544f0aee703 P 1c708c44654e461099098050b30d16f0: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260807 15:11:58.453748 9304 heartbeater.cc:503] Master 127.29.194.62:34237 was elected leader, sending a full tablet report...
I20260807 15:11:58.454612 9759 tablet_bootstrap.cc:492] T f79d6eba55f347adbd2e7544f0aee703 P 7c8b57fd081d41bb8f74cf365d3e0a7a: Bootstrap starting.
I20260807 15:11:58.455219 9759 tablet_bootstrap.cc:654] T f79d6eba55f347adbd2e7544f0aee703 P 7c8b57fd081d41bb8f74cf365d3e0a7a: Neither blocks nor log segments found. Creating new log.
I20260807 15:11:58.455525 9759 log.cc:824] T f79d6eba55f347adbd2e7544f0aee703 P 7c8b57fd081d41bb8f74cf365d3e0a7a: Log is configured to *not* fsync() on all Append() calls
I20260807 15:11:58.456290 9759 tablet_bootstrap.cc:492] T f79d6eba55f347adbd2e7544f0aee703 P 7c8b57fd081d41bb8f74cf365d3e0a7a: No bootstrap required, opened a new log
I20260807 15:11:58.456400 9759 ts_tablet_manager.cc:1397] T f79d6eba55f347adbd2e7544f0aee703 P 7c8b57fd081d41bb8f74cf365d3e0a7a: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 15:11:58.457994 9759 raft_consensus.cc:348] T f79d6eba55f347adbd2e7544f0aee703 P 7c8b57fd081d41bb8f74cf365d3e0a7a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c708c44654e461099098050b30d16f0" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 36541 } } peers { permanent_uuid: "86b01224f5b34c19a27d23a31ab7306a" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 44857 } } peers { permanent_uuid: "7c8b57fd081d41bb8f74cf365d3e0a7a" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 36405 } }
I20260807 15:11:58.458165 9759 raft_consensus.cc:374] T f79d6eba55f347adbd2e7544f0aee703 P 7c8b57fd081d41bb8f74cf365d3e0a7a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:11:58.458197 9759 raft_consensus.cc:729] T f79d6eba55f347adbd2e7544f0aee703 P 7c8b57fd081d41bb8f74cf365d3e0a7a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c8b57fd081d41bb8f74cf365d3e0a7a, State: Initialized, Role: FOLLOWER
I20260807 15:11:58.458305 9759 consensus_queue.cc:260] T f79d6eba55f347adbd2e7544f0aee703 P 7c8b57fd081d41bb8f74cf365d3e0a7a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c708c44654e461099098050b30d16f0" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 36541 } } peers { permanent_uuid: "86b01224f5b34c19a27d23a31ab7306a" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 44857 } } peers { permanent_uuid: "7c8b57fd081d41bb8f74cf365d3e0a7a" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 36405 } }
I20260807 15:11:58.458611 9759 ts_tablet_manager.cc:1428] T f79d6eba55f347adbd2e7544f0aee703 P 7c8b57fd081d41bb8f74cf365d3e0a7a: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260807 15:11:58.458684 9506 heartbeater.cc:503] Master 127.29.194.62:34237 was elected leader, sending a full tablet report...
W20260807 15:11:58.458727 9507 tablet.cc:2369] T f79d6eba55f347adbd2e7544f0aee703 P 7c8b57fd081d41bb8f74cf365d3e0a7a: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 15:11:58.533947 9305 tablet.cc:2369] T f79d6eba55f347adbd2e7544f0aee703 P 1c708c44654e461099098050b30d16f0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 15:11:58.635190 9758 raft_consensus.cc:482] T f79d6eba55f347adbd2e7544f0aee703 P 1c708c44654e461099098050b30d16f0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 15:11:58.635479 9758 raft_consensus.cc:504] T f79d6eba55f347adbd2e7544f0aee703 P 1c708c44654e461099098050b30d16f0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c708c44654e461099098050b30d16f0" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 36541 } } peers { permanent_uuid: "86b01224f5b34c19a27d23a31ab7306a" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 44857 } } peers { permanent_uuid: "7c8b57fd081d41bb8f74cf365d3e0a7a" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 36405 } }
I20260807 15:11:58.636073 9758 leader_election.cc:288] T f79d6eba55f347adbd2e7544f0aee703 P 1c708c44654e461099098050b30d16f0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 86b01224f5b34c19a27d23a31ab7306a (127.29.194.3:44857), 7c8b57fd081d41bb8f74cf365d3e0a7a (127.29.194.2:36405)
W20260807 15:11:58.639118 9711 tablet.cc:2369] T f79d6eba55f347adbd2e7544f0aee703 P 86b01224f5b34c19a27d23a31ab7306a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 15:11:58.640633 9742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:58.636465 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.194.1:50305 (local address 127.29.194.2:36405)
0807 15:11:58.636531 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:58.636535 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:58.636701 (+ 166us) server_negotiation.cc:407] Connection header received
0807 15:11:58.636776 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:58.636778 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:58.636788 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:58.636870 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:11:58.637182 (+ 312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:58.637986 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:58.639605 (+ 1619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:58.640180 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:58.640262 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:58.640271 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0807 15:11:58.640519 (+ 248us) server_negotiation.cc:299] Negotiation successful
0807 15:11:58.640560 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260807 15:11:58.640633 9748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:58.636499 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.194.1:58947 (local address 127.29.194.3:44857)
0807 15:11:58.636600 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:58.636602 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:58.636701 (+ 99us) server_negotiation.cc:407] Connection header received
0807 15:11:58.636796 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:58.636798 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:58.636806 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:58.636870 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:11:58.637183 (+ 313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:58.637974 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:58.639605 (+ 1631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:58.640180 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:58.640261 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:58.640271 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0807 15:11:58.640519 (+ 248us) server_negotiation.cc:299] Negotiation successful
0807 15:11:58.640560 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260807 15:11:58.640684 9764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:58.636366 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.3:44857 (local address 127.29.194.1:58947)
0807 15:11:58.636591 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0807 15:11:58.636605 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 15:11:58.636701 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:11:58.636960 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:11:58.636962 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:11:58.636967 (+ 5us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:11:58.637090 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:58.637092 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:58.638099 (+ 1007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:58.638101 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:58.639479 (+ 1378us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:58.639484 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:58.640327 (+ 843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:58.640328 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:58.640432 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:58.640438 (+ 6us) client_negotiation.cc:769] Sending connection context
0807 15:11:58.640507 (+ 69us) client_negotiation.cc:240] Negotiation successful
0807 15:11:58.640568 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":201,"mutex_wait_us":84,"thread_start_us":105,"threads_started":1}
I20260807 15:11:58.640689 9763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:58.636375 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.2:36405 (local address 127.29.194.1:50305)
0807 15:11:58.636591 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0807 15:11:58.636605 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 15:11:58.636701 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:11:58.636960 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:11:58.636962 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:11:58.636977 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:11:58.637089 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:58.637092 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:58.638101 (+ 1009us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:58.638101 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:58.639481 (+ 1380us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:58.639484 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:58.640329 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:58.640330 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:58.640432 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:58.640440 (+ 8us) client_negotiation.cc:769] Sending connection context
0807 15:11:58.640507 (+ 67us) client_negotiation.cc:240] Negotiation successful
0807 15:11:58.640568 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":198,"mutex_wait_us":97,"thread_start_us":88,"threads_started":1}
I20260807 15:11:58.641152 9636 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f79d6eba55f347adbd2e7544f0aee703" candidate_uuid: "1c708c44654e461099098050b30d16f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "86b01224f5b34c19a27d23a31ab7306a" is_pre_election: true
I20260807 15:11:58.641156 9437 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f79d6eba55f347adbd2e7544f0aee703" candidate_uuid: "1c708c44654e461099098050b30d16f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c8b57fd081d41bb8f74cf365d3e0a7a" is_pre_election: true
I20260807 15:11:58.641310 9636 raft_consensus.cc:2454] T f79d6eba55f347adbd2e7544f0aee703 P 86b01224f5b34c19a27d23a31ab7306a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1c708c44654e461099098050b30d16f0 in term 0.
I20260807 15:11:58.641310 9437 raft_consensus.cc:2454] T f79d6eba55f347adbd2e7544f0aee703 P 7c8b57fd081d41bb8f74cf365d3e0a7a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1c708c44654e461099098050b30d16f0 in term 0.
I20260807 15:11:58.641600 9149 leader_election.cc:302] T f79d6eba55f347adbd2e7544f0aee703 P 1c708c44654e461099098050b30d16f0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1c708c44654e461099098050b30d16f0, 86b01224f5b34c19a27d23a31ab7306a; no voters:
I20260807 15:11:58.641790 9758 raft_consensus.cc:2781] T f79d6eba55f347adbd2e7544f0aee703 P 1c708c44654e461099098050b30d16f0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 15:11:58.641834 9758 raft_consensus.cc:482] T f79d6eba55f347adbd2e7544f0aee703 P 1c708c44654e461099098050b30d16f0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 15:11:58.641850 9758 raft_consensus.cc:3037] T f79d6eba55f347adbd2e7544f0aee703 P 1c708c44654e461099098050b30d16f0 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:11:58.642654 9758 raft_consensus.cc:504] T f79d6eba55f347adbd2e7544f0aee703 P 1c708c44654e461099098050b30d16f0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c708c44654e461099098050b30d16f0" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 36541 } } peers { permanent_uuid: "86b01224f5b34c19a27d23a31ab7306a" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 44857 } } peers { permanent_uuid: "7c8b57fd081d41bb8f74cf365d3e0a7a" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 36405 } }
I20260807 15:11:58.642796 9758 leader_election.cc:288] T f79d6eba55f347adbd2e7544f0aee703 P 1c708c44654e461099098050b30d16f0 [CANDIDATE]: Term 1 election: Requested vote from peers 86b01224f5b34c19a27d23a31ab7306a (127.29.194.3:44857), 7c8b57fd081d41bb8f74cf365d3e0a7a (127.29.194.2:36405)
I20260807 15:11:58.642990 9636 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f79d6eba55f347adbd2e7544f0aee703" candidate_uuid: "1c708c44654e461099098050b30d16f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "86b01224f5b34c19a27d23a31ab7306a"
I20260807 15:11:58.643028 9437 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f79d6eba55f347adbd2e7544f0aee703" candidate_uuid: "1c708c44654e461099098050b30d16f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c8b57fd081d41bb8f74cf365d3e0a7a"
I20260807 15:11:58.643070 9636 raft_consensus.cc:3037] T f79d6eba55f347adbd2e7544f0aee703 P 86b01224f5b34c19a27d23a31ab7306a [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:11:58.643107 9437 raft_consensus.cc:3037] T f79d6eba55f347adbd2e7544f0aee703 P 7c8b57fd081d41bb8f74cf365d3e0a7a [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:11:58.643805 9636 raft_consensus.cc:2454] T f79d6eba55f347adbd2e7544f0aee703 P 86b01224f5b34c19a27d23a31ab7306a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1c708c44654e461099098050b30d16f0 in term 1.
I20260807 15:11:58.643828 9437 raft_consensus.cc:2454] T f79d6eba55f347adbd2e7544f0aee703 P 7c8b57fd081d41bb8f74cf365d3e0a7a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1c708c44654e461099098050b30d16f0 in term 1.
I20260807 15:11:58.643997 9151 leader_election.cc:302] T f79d6eba55f347adbd2e7544f0aee703 P 1c708c44654e461099098050b30d16f0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1c708c44654e461099098050b30d16f0, 7c8b57fd081d41bb8f74cf365d3e0a7a; no voters:
I20260807 15:11:58.644100 9758 raft_consensus.cc:2781] T f79d6eba55f347adbd2e7544f0aee703 P 1c708c44654e461099098050b30d16f0 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:11:58.644236 9758 raft_consensus.cc:686] T f79d6eba55f347adbd2e7544f0aee703 P 1c708c44654e461099098050b30d16f0 [term 1 LEADER]: Becoming Leader. State: Replica: 1c708c44654e461099098050b30d16f0, State: Running, Role: LEADER
I20260807 15:11:58.644335 9758 consensus_queue.cc:237] T f79d6eba55f347adbd2e7544f0aee703 P 1c708c44654e461099098050b30d16f0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c708c44654e461099098050b30d16f0" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 36541 } } peers { permanent_uuid: "86b01224f5b34c19a27d23a31ab7306a" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 44857 } } peers { permanent_uuid: "7c8b57fd081d41bb8f74cf365d3e0a7a" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 36405 } }
I20260807 15:11:58.645591 9009 catalog_manager.cc:5495] T f79d6eba55f347adbd2e7544f0aee703 P 1c708c44654e461099098050b30d16f0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1c708c44654e461099098050b30d16f0 (127.29.194.1). New cstate: current_term: 1 leader_uuid: "1c708c44654e461099098050b30d16f0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c708c44654e461099098050b30d16f0" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 36541 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "86b01224f5b34c19a27d23a31ab7306a" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 44857 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7c8b57fd081d41bb8f74cf365d3e0a7a" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 36405 } health_report { overall_health: UNKNOWN } } }
I20260807 15:11:58.674821 9745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:58.671887 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34048 (local address 127.29.194.1:36541)
0807 15:11:58.671945 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:58.671947 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:58.671957 (+ 10us) server_negotiation.cc:407] Connection header received
0807 15:11:58.672001 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:58.672002 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:58.672020 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:58.672086 (+ 66us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 15:11:58.672199 (+ 113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:58.672698 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:58.673814 (+ 1116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:58.674040 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:58.674119 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:58.674418 (+ 299us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 15:11:58.674537 (+ 119us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 15:11:58.674612 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0807 15:11:58.674724 (+ 112us) server_negotiation.cc:299] Negotiation successful
0807 15:11:58.674749 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260807 15:11:58.677000 9636 raft_consensus.cc:1261] T f79d6eba55f347adbd2e7544f0aee703 P 86b01224f5b34c19a27d23a31ab7306a [term 1 FOLLOWER]: Refusing update from remote peer 1c708c44654e461099098050b30d16f0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:11:58.676992 9437 raft_consensus.cc:1261] T f79d6eba55f347adbd2e7544f0aee703 P 7c8b57fd081d41bb8f74cf365d3e0a7a [term 1 FOLLOWER]: Refusing update from remote peer 1c708c44654e461099098050b30d16f0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:11:58.677387 9758 consensus_queue.cc:1035] T f79d6eba55f347adbd2e7544f0aee703 P 1c708c44654e461099098050b30d16f0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "86b01224f5b34c19a27d23a31ab7306a" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 44857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:11:58.677842 9765 consensus_queue.cc:1035] T f79d6eba55f347adbd2e7544f0aee703 P 1c708c44654e461099098050b30d16f0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c8b57fd081d41bb8f74cf365d3e0a7a" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 36405 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:11:58.681121 9774 mvcc.cc:205] Tried to move back new op lower bound from 7315929164497743872 to 7315929164367912960. Current Snapshot: MvccSnapshot[applied={T|T < 7315929164497743872}]
I20260807 15:11:58.681219 9771 mvcc.cc:205] Tried to move back new op lower bound from 7315929164497743872 to 7315929164367912960. Current Snapshot: MvccSnapshot[applied={T|T < 7315929164497743872 or (T in {7315929164497743872})}]
I20260807 15:11:58.683192 9009 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56732:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 15:11:58.683307 9009 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56732:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 15:11:58.684342 9009 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f743aff5796d4ca99c67bbf7b9b722d6: Sending DeleteTablet for 3 replicas of tablet f79d6eba55f347adbd2e7544f0aee703
I20260807 15:11:58.684756 9612 tablet_service.cc:1514] Processing DeleteTablet for tablet f79d6eba55f347adbd2e7544f0aee703 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:11:58 UTC) from {username='oryx', principal='oryx/127.29.194.62@KRBTEST.COM'} at 127.0.0.1:46368
I20260807 15:11:58.684792 9410 tablet_service.cc:1514] Processing DeleteTablet for tablet f79d6eba55f347adbd2e7544f0aee703 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:11:58 UTC) from {username='oryx', principal='oryx/127.29.194.62@KRBTEST.COM'} at 127.0.0.1:35692
I20260807 15:11:58.684839 9214 tablet_service.cc:1514] Processing DeleteTablet for tablet f79d6eba55f347adbd2e7544f0aee703 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:11:58 UTC) from {username='oryx', principal='oryx/127.29.194.62@KRBTEST.COM'} at 127.0.0.1:34040
I20260807 15:11:58.685019 9009 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56732:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 15:11:58.685178 9009 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 15:11:58.685264 9784 tablet_replica.cc:330] T f79d6eba55f347adbd2e7544f0aee703 P 7c8b57fd081d41bb8f74cf365d3e0a7a: stopping tablet replica
I20260807 15:11:58.685371 9784 raft_consensus.cc:2229] T f79d6eba55f347adbd2e7544f0aee703 P 7c8b57fd081d41bb8f74cf365d3e0a7a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 15:11:58.685561 9784 raft_consensus.cc:2258] T f79d6eba55f347adbd2e7544f0aee703 P 7c8b57fd081d41bb8f74cf365d3e0a7a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:11:58.685734 9783 tablet_replica.cc:330] T f79d6eba55f347adbd2e7544f0aee703 P 86b01224f5b34c19a27d23a31ab7306a: stopping tablet replica
I20260807 15:11:58.685844 9783 raft_consensus.cc:2229] T f79d6eba55f347adbd2e7544f0aee703 P 86b01224f5b34c19a27d23a31ab7306a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 15:11:58.686195 9783 raft_consensus.cc:2258] T f79d6eba55f347adbd2e7544f0aee703 P 86b01224f5b34c19a27d23a31ab7306a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:11:58.686712 9214 tablet_service.cc:1467] Processing CreateTablet for tablet 91f6d1ab0a5b4a73a55e3f82d309e53e (DEFAULT_TABLE table=test-table [id=0a2c77d43c7e4d46b2bd3a64fbb92336]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:11:58.686723 9612 tablet_service.cc:1467] Processing CreateTablet for tablet 91f6d1ab0a5b4a73a55e3f82d309e53e (DEFAULT_TABLE table=test-table [id=0a2c77d43c7e4d46b2bd3a64fbb92336]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:11:58.686865 9214 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 91f6d1ab0a5b4a73a55e3f82d309e53e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:11:58.686868 9612 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 91f6d1ab0a5b4a73a55e3f82d309e53e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:11:58.688057 9750 tablet_bootstrap.cc:492] T 91f6d1ab0a5b4a73a55e3f82d309e53e P 1c708c44654e461099098050b30d16f0: Bootstrap starting.
I20260807 15:11:58.688326 9749 tablet_bootstrap.cc:492] T 91f6d1ab0a5b4a73a55e3f82d309e53e P 86b01224f5b34c19a27d23a31ab7306a: Bootstrap starting.
I20260807 15:11:58.688409 9410 tablet_service.cc:1467] Processing CreateTablet for tablet 91f6d1ab0a5b4a73a55e3f82d309e53e (DEFAULT_TABLE table=test-table [id=0a2c77d43c7e4d46b2bd3a64fbb92336]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:11:58.688517 9750 tablet_bootstrap.cc:654] T 91f6d1ab0a5b4a73a55e3f82d309e53e P 1c708c44654e461099098050b30d16f0: Neither blocks nor log segments found. Creating new log.
I20260807 15:11:58.688546 9410 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 91f6d1ab0a5b4a73a55e3f82d309e53e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:11:58.688746 9749 tablet_bootstrap.cc:654] T 91f6d1ab0a5b4a73a55e3f82d309e53e P 86b01224f5b34c19a27d23a31ab7306a: Neither blocks nor log segments found. Creating new log.
I20260807 15:11:58.689306 9749 tablet_bootstrap.cc:492] T 91f6d1ab0a5b4a73a55e3f82d309e53e P 86b01224f5b34c19a27d23a31ab7306a: No bootstrap required, opened a new log
I20260807 15:11:58.689308 9784 ts_tablet_manager.cc:1905] T f79d6eba55f347adbd2e7544f0aee703 P 7c8b57fd081d41bb8f74cf365d3e0a7a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:11:58.689356 9749 ts_tablet_manager.cc:1397] T 91f6d1ab0a5b4a73a55e3f82d309e53e P 86b01224f5b34c19a27d23a31ab7306a: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260807 15:11:58.689632 9749 raft_consensus.cc:348] T 91f6d1ab0a5b4a73a55e3f82d309e53e P 86b01224f5b34c19a27d23a31ab7306a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c708c44654e461099098050b30d16f0" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 36541 } } peers { permanent_uuid: "7c8b57fd081d41bb8f74cf365d3e0a7a" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 36405 } } peers { permanent_uuid: "86b01224f5b34c19a27d23a31ab7306a" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 44857 } }
I20260807 15:11:58.689714 9749 raft_consensus.cc:374] T 91f6d1ab0a5b4a73a55e3f82d309e53e P 86b01224f5b34c19a27d23a31ab7306a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:11:58.689734 9749 raft_consensus.cc:729] T 91f6d1ab0a5b4a73a55e3f82d309e53e P 86b01224f5b34c19a27d23a31ab7306a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 86b01224f5b34c19a27d23a31ab7306a, State: Initialized, Role: FOLLOWER
I20260807 15:11:58.689772 9749 consensus_queue.cc:260] T 91f6d1ab0a5b4a73a55e3f82d309e53e P 86b01224f5b34c19a27d23a31ab7306a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c708c44654e461099098050b30d16f0" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 36541 } } peers { permanent_uuid: "7c8b57fd081d41bb8f74cf365d3e0a7a" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 36405 } } peers { permanent_uuid: "86b01224f5b34c19a27d23a31ab7306a" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 44857 } }
I20260807 15:11:58.689851 9749 ts_tablet_manager.cc:1428] T 91f6d1ab0a5b4a73a55e3f82d309e53e P 86b01224f5b34c19a27d23a31ab7306a: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260807 15:11:58.690019 9759 tablet_bootstrap.cc:492] T 91f6d1ab0a5b4a73a55e3f82d309e53e P 7c8b57fd081d41bb8f74cf365d3e0a7a: Bootstrap starting.
I20260807 15:11:58.689981 9783 ts_tablet_manager.cc:1905] T f79d6eba55f347adbd2e7544f0aee703 P 86b01224f5b34c19a27d23a31ab7306a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:11:58.690446 9759 tablet_bootstrap.cc:654] T 91f6d1ab0a5b4a73a55e3f82d309e53e P 7c8b57fd081d41bb8f74cf365d3e0a7a: Neither blocks nor log segments found. Creating new log.
I20260807 15:11:58.691326 9759 tablet_bootstrap.cc:492] T 91f6d1ab0a5b4a73a55e3f82d309e53e P 7c8b57fd081d41bb8f74cf365d3e0a7a: No bootstrap required, opened a new log
I20260807 15:11:58.691387 9759 ts_tablet_manager.cc:1397] T 91f6d1ab0a5b4a73a55e3f82d309e53e P 7c8b57fd081d41bb8f74cf365d3e0a7a: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260807 15:11:58.691551 9759 raft_consensus.cc:348] T 91f6d1ab0a5b4a73a55e3f82d309e53e P 7c8b57fd081d41bb8f74cf365d3e0a7a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c708c44654e461099098050b30d16f0" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 36541 } } peers { permanent_uuid: "7c8b57fd081d41bb8f74cf365d3e0a7a" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 36405 } } peers { permanent_uuid: "86b01224f5b34c19a27d23a31ab7306a" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 44857 } }
I20260807 15:11:58.691642 9759 raft_consensus.cc:374] T 91f6d1ab0a5b4a73a55e3f82d309e53e P 7c8b57fd081d41bb8f74cf365d3e0a7a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:11:58.691665 9759 raft_consensus.cc:729] T 91f6d1ab0a5b4a73a55e3f82d309e53e P 7c8b57fd081d41bb8f74cf365d3e0a7a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c8b57fd081d41bb8f74cf365d3e0a7a, State: Initialized, Role: FOLLOWER
I20260807 15:11:58.691710 9759 consensus_queue.cc:260] T 91f6d1ab0a5b4a73a55e3f82d309e53e P 7c8b57fd081d41bb8f74cf365d3e0a7a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c708c44654e461099098050b30d16f0" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 36541 } } peers { permanent_uuid: "7c8b57fd081d41bb8f74cf365d3e0a7a" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 36405 } } peers { permanent_uuid: "86b01224f5b34c19a27d23a31ab7306a" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 44857 } }
I20260807 15:11:58.691800 9759 ts_tablet_manager.cc:1428] T 91f6d1ab0a5b4a73a55e3f82d309e53e P 7c8b57fd081d41bb8f74cf365d3e0a7a: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260807 15:11:58.692060 9783 ts_tablet_manager.cc:1918] T f79d6eba55f347adbd2e7544f0aee703 P 86b01224f5b34c19a27d23a31ab7306a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 15:11:58.692133 9786 tablet_replica.cc:330] T f79d6eba55f347adbd2e7544f0aee703 P 1c708c44654e461099098050b30d16f0: stopping tablet replica
I20260807 15:11:58.692230 9783 log.cc:1196] T f79d6eba55f347adbd2e7544f0aee703 P 86b01224f5b34c19a27d23a31ab7306a: Deleting WAL directory at /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115480851507-30472-0/minicluster-data/ts-2/wal/wals/f79d6eba55f347adbd2e7544f0aee703
I20260807 15:11:58.692278 9786 raft_consensus.cc:2229] T f79d6eba55f347adbd2e7544f0aee703 P 1c708c44654e461099098050b30d16f0 [term 1 LEADER]: Raft consensus shutting down.
I20260807 15:11:58.692406 9786 raft_consensus.cc:2258] T f79d6eba55f347adbd2e7544f0aee703 P 1c708c44654e461099098050b30d16f0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:11:58.692734 9783 ts_tablet_manager.cc:1939] T f79d6eba55f347adbd2e7544f0aee703 P 86b01224f5b34c19a27d23a31ab7306a: Deleting consensus metadata
I20260807 15:11:58.693185 9784 ts_tablet_manager.cc:1918] T f79d6eba55f347adbd2e7544f0aee703 P 7c8b57fd081d41bb8f74cf365d3e0a7a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 15:11:58.693264 9784 log.cc:1196] T f79d6eba55f347adbd2e7544f0aee703 P 7c8b57fd081d41bb8f74cf365d3e0a7a: Deleting WAL directory at /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115480851507-30472-0/minicluster-data/ts-1/wal/wals/f79d6eba55f347adbd2e7544f0aee703
I20260807 15:11:58.693238 8985 catalog_manager.cc:4841] TS 86b01224f5b34c19a27d23a31ab7306a (127.29.194.3:44857): tablet f79d6eba55f347adbd2e7544f0aee703 (table test-table [id=d7dd955011f64e49a98093a94cbb58b5]) successfully deleted
I20260807 15:11:58.693550 9784 ts_tablet_manager.cc:1939] T f79d6eba55f347adbd2e7544f0aee703 P 7c8b57fd081d41bb8f74cf365d3e0a7a: Deleting consensus metadata
I20260807 15:11:58.693948 8986 catalog_manager.cc:4841] TS 7c8b57fd081d41bb8f74cf365d3e0a7a (127.29.194.2:36405): tablet f79d6eba55f347adbd2e7544f0aee703 (table test-table [id=d7dd955011f64e49a98093a94cbb58b5]) successfully deleted
I20260807 15:11:58.694235 9750 tablet_bootstrap.cc:492] T 91f6d1ab0a5b4a73a55e3f82d309e53e P 1c708c44654e461099098050b30d16f0: No bootstrap required, opened a new log
I20260807 15:11:58.694288 9750 ts_tablet_manager.cc:1397] T 91f6d1ab0a5b4a73a55e3f82d309e53e P 1c708c44654e461099098050b30d16f0: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.001s
I20260807 15:11:58.694427 9750 raft_consensus.cc:348] T 91f6d1ab0a5b4a73a55e3f82d309e53e P 1c708c44654e461099098050b30d16f0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c708c44654e461099098050b30d16f0" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 36541 } } peers { permanent_uuid: "7c8b57fd081d41bb8f74cf365d3e0a7a" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 36405 } } peers { permanent_uuid: "86b01224f5b34c19a27d23a31ab7306a" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 44857 } }
I20260807 15:11:58.694494 9750 raft_consensus.cc:374] T 91f6d1ab0a5b4a73a55e3f82d309e53e P 1c708c44654e461099098050b30d16f0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:11:58.694507 9750 raft_consensus.cc:729] T 91f6d1ab0a5b4a73a55e3f82d309e53e P 1c708c44654e461099098050b30d16f0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1c708c44654e461099098050b30d16f0, State: Initialized, Role: FOLLOWER
I20260807 15:11:58.694545 9750 consensus_queue.cc:260] T 91f6d1ab0a5b4a73a55e3f82d309e53e P 1c708c44654e461099098050b30d16f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c708c44654e461099098050b30d16f0" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 36541 } } peers { permanent_uuid: "7c8b57fd081d41bb8f74cf365d3e0a7a" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 36405 } } peers { permanent_uuid: "86b01224f5b34c19a27d23a31ab7306a" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 44857 } }
I20260807 15:11:58.694633 9750 ts_tablet_manager.cc:1428] T 91f6d1ab0a5b4a73a55e3f82d309e53e P 1c708c44654e461099098050b30d16f0: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260807 15:11:58.698365 9786 ts_tablet_manager.cc:1905] T f79d6eba55f347adbd2e7544f0aee703 P 1c708c44654e461099098050b30d16f0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:11:58.700331 9786 ts_tablet_manager.cc:1918] T f79d6eba55f347adbd2e7544f0aee703 P 1c708c44654e461099098050b30d16f0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 15:11:58.700428 9786 log.cc:1196] T f79d6eba55f347adbd2e7544f0aee703 P 1c708c44654e461099098050b30d16f0: Deleting WAL directory at /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115480851507-30472-0/minicluster-data/ts-0/wal/wals/f79d6eba55f347adbd2e7544f0aee703
I20260807 15:11:58.700716 9786 ts_tablet_manager.cc:1939] T f79d6eba55f347adbd2e7544f0aee703 P 1c708c44654e461099098050b30d16f0: Deleting consensus metadata
I20260807 15:11:58.701318 8985 catalog_manager.cc:4841] TS 1c708c44654e461099098050b30d16f0 (127.29.194.1:36541): tablet f79d6eba55f347adbd2e7544f0aee703 (table test-table [id=d7dd955011f64e49a98093a94cbb58b5]) successfully deleted
I20260807 15:11:58.801625 9755 raft_consensus.cc:482] T 91f6d1ab0a5b4a73a55e3f82d309e53e P 86b01224f5b34c19a27d23a31ab7306a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 15:11:58.801759 9755 raft_consensus.cc:504] T 91f6d1ab0a5b4a73a55e3f82d309e53e P 86b01224f5b34c19a27d23a31ab7306a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c708c44654e461099098050b30d16f0" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 36541 } } peers { permanent_uuid: "7c8b57fd081d41bb8f74cf365d3e0a7a" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 36405 } } peers { permanent_uuid: "86b01224f5b34c19a27d23a31ab7306a" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 44857 } }
I20260807 15:11:58.802109 9755 leader_election.cc:288] T 91f6d1ab0a5b4a73a55e3f82d309e53e P 86b01224f5b34c19a27d23a31ab7306a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1c708c44654e461099098050b30d16f0 (127.29.194.1:36541), 7c8b57fd081d41bb8f74cf365d3e0a7a (127.29.194.2:36405)
I20260807 15:11:58.806241 9561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:58.802468 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.2:36405 (local address 127.29.194.3:58005)
0807 15:11:58.802620 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0807 15:11:58.802641 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 15:11:58.802737 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:11:58.802947 (+ 210us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:11:58.802949 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:11:58.802951 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:11:58.803041 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:58.803042 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:58.804021 (+ 979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:58.804021 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:58.805335 (+ 1314us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:58.805339 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:58.805950 (+ 611us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:58.805951 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:58.806055 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:58.806058 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 15:11:58.806108 (+ 50us) client_negotiation.cc:240] Negotiation successful
0807 15:11:58.806159 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"mutex_wait_us":59}
I20260807 15:11:58.806241 9742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:58.802714 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.194.3:58005 (local address 127.29.194.2:36405)
0807 15:11:58.802803 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:58.802806 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:58.802814 (+ 8us) server_negotiation.cc:407] Connection header received
0807 15:11:58.802845 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:58.802846 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:58.802855 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:58.802914 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:11:58.803151 (+ 237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:58.803928 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:58.805497 (+ 1569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:58.805867 (+ 370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:58.805935 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:58.805952 (+ 17us) server_negotiation.cc:1035] Waiting for connection context
0807 15:11:58.806132 (+ 180us) server_negotiation.cc:299] Negotiation successful
0807 15:11:58.806175 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260807 15:11:58.806461 9745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:58.802611 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.194.3:45959 (local address 127.29.194.1:36541)
0807 15:11:58.802661 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:58.802663 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:58.803002 (+ 339us) server_negotiation.cc:407] Connection header received
0807 15:11:58.803038 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:58.803039 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:58.803050 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:58.803113 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:11:58.803336 (+ 223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:58.803905 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:58.805467 (+ 1562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:58.806050 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:58.806122 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:58.806130 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0807 15:11:58.806352 (+ 222us) server_negotiation.cc:299] Negotiation successful
0807 15:11:58.806387 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260807 15:11:58.806531 9437 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "91f6d1ab0a5b4a73a55e3f82d309e53e" candidate_uuid: "86b01224f5b34c19a27d23a31ab7306a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c8b57fd081d41bb8f74cf365d3e0a7a" is_pre_election: true
I20260807 15:11:58.806638 9437 raft_consensus.cc:2454] T 91f6d1ab0a5b4a73a55e3f82d309e53e P 7c8b57fd081d41bb8f74cf365d3e0a7a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 86b01224f5b34c19a27d23a31ab7306a in term 0.
I20260807 15:11:58.806715 9791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:58.802468 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.1:36541 (local address 127.29.194.3:45959)
0807 15:11:58.802927 (+ 459us) negotiation.cc:107] Waiting for socket to connect
0807 15:11:58.802937 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 15:11:58.802994 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:11:58.803135 (+ 141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:11:58.803136 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:11:58.803138 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:11:58.803212 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:58.803214 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:58.803987 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:58.803989 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:58.805335 (+ 1346us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:58.805339 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:58.806199 (+ 860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:58.806200 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:58.806276 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:58.806280 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 15:11:58.806582 (+ 302us) client_negotiation.cc:240] Negotiation successful
0807 15:11:58.806630 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":445,"mutex_wait_us":359,"thread_start_us":72,"threads_started":1}
I20260807 15:11:58.806821 9551 leader_election.cc:302] T 91f6d1ab0a5b4a73a55e3f82d309e53e P 86b01224f5b34c19a27d23a31ab7306a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c8b57fd081d41bb8f74cf365d3e0a7a, 86b01224f5b34c19a27d23a31ab7306a; no voters:
I20260807 15:11:58.806967 9755 raft_consensus.cc:2781] T 91f6d1ab0a5b4a73a55e3f82d309e53e P 86b01224f5b34c19a27d23a31ab7306a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 15:11:58.807004 9755 raft_consensus.cc:482] T 91f6d1ab0a5b4a73a55e3f82d309e53e P 86b01224f5b34c19a27d23a31ab7306a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 15:11:58.807029 9755 raft_consensus.cc:3037] T 91f6d1ab0a5b4a73a55e3f82d309e53e P 86b01224f5b34c19a27d23a31ab7306a [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:11:58.806949 9242 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "91f6d1ab0a5b4a73a55e3f82d309e53e" candidate_uuid: "86b01224f5b34c19a27d23a31ab7306a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c708c44654e461099098050b30d16f0" is_pre_election: true
I20260807 15:11:58.807089 9242 raft_consensus.cc:2454] T 91f6d1ab0a5b4a73a55e3f82d309e53e P 1c708c44654e461099098050b30d16f0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 86b01224f5b34c19a27d23a31ab7306a in term 0.
I20260807 15:11:58.807894 9755 raft_consensus.cc:504] T 91f6d1ab0a5b4a73a55e3f82d309e53e P 86b01224f5b34c19a27d23a31ab7306a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c708c44654e461099098050b30d16f0" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 36541 } } peers { permanent_uuid: "7c8b57fd081d41bb8f74cf365d3e0a7a" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 36405 } } peers { permanent_uuid: "86b01224f5b34c19a27d23a31ab7306a" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 44857 } }
I20260807 15:11:58.808056 9755 leader_election.cc:288] T 91f6d1ab0a5b4a73a55e3f82d309e53e P 86b01224f5b34c19a27d23a31ab7306a [CANDIDATE]: Term 1 election: Requested vote from peers 1c708c44654e461099098050b30d16f0 (127.29.194.1:36541), 7c8b57fd081d41bb8f74cf365d3e0a7a (127.29.194.2:36405)
I20260807 15:11:58.808202 9242 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "91f6d1ab0a5b4a73a55e3f82d309e53e" candidate_uuid: "86b01224f5b34c19a27d23a31ab7306a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c708c44654e461099098050b30d16f0"
I20260807 15:11:58.808234 9437 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "91f6d1ab0a5b4a73a55e3f82d309e53e" candidate_uuid: "86b01224f5b34c19a27d23a31ab7306a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c8b57fd081d41bb8f74cf365d3e0a7a"
I20260807 15:11:58.808270 9242 raft_consensus.cc:3037] T 91f6d1ab0a5b4a73a55e3f82d309e53e P 1c708c44654e461099098050b30d16f0 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:11:58.808298 9437 raft_consensus.cc:3037] T 91f6d1ab0a5b4a73a55e3f82d309e53e P 7c8b57fd081d41bb8f74cf365d3e0a7a [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:11:58.809054 9242 raft_consensus.cc:2454] T 91f6d1ab0a5b4a73a55e3f82d309e53e P 1c708c44654e461099098050b30d16f0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 86b01224f5b34c19a27d23a31ab7306a in term 1.
I20260807 15:11:58.809054 9437 raft_consensus.cc:2454] T 91f6d1ab0a5b4a73a55e3f82d309e53e P 7c8b57fd081d41bb8f74cf365d3e0a7a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 86b01224f5b34c19a27d23a31ab7306a in term 1.
I20260807 15:11:58.809255 9550 leader_election.cc:302] T 91f6d1ab0a5b4a73a55e3f82d309e53e P 86b01224f5b34c19a27d23a31ab7306a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1c708c44654e461099098050b30d16f0, 86b01224f5b34c19a27d23a31ab7306a; no voters:
I20260807 15:11:58.809350 9755 raft_consensus.cc:2781] T 91f6d1ab0a5b4a73a55e3f82d309e53e P 86b01224f5b34c19a27d23a31ab7306a [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:11:58.809568 9755 raft_consensus.cc:686] T 91f6d1ab0a5b4a73a55e3f82d309e53e P 86b01224f5b34c19a27d23a31ab7306a [term 1 LEADER]: Becoming Leader. State: Replica: 86b01224f5b34c19a27d23a31ab7306a, State: Running, Role: LEADER
I20260807 15:11:58.809655 9755 consensus_queue.cc:237] T 91f6d1ab0a5b4a73a55e3f82d309e53e P 86b01224f5b34c19a27d23a31ab7306a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c708c44654e461099098050b30d16f0" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 36541 } } peers { permanent_uuid: "7c8b57fd081d41bb8f74cf365d3e0a7a" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 36405 } } peers { permanent_uuid: "86b01224f5b34c19a27d23a31ab7306a" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 44857 } }
I20260807 15:11:58.810400 9008 catalog_manager.cc:5495] T 91f6d1ab0a5b4a73a55e3f82d309e53e P 86b01224f5b34c19a27d23a31ab7306a reported cstate change: term changed from 0 to 1, leader changed from <none> to 86b01224f5b34c19a27d23a31ab7306a (127.29.194.3). New cstate: current_term: 1 leader_uuid: "86b01224f5b34c19a27d23a31ab7306a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c708c44654e461099098050b30d16f0" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 36541 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7c8b57fd081d41bb8f74cf365d3e0a7a" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 36405 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "86b01224f5b34c19a27d23a31ab7306a" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 44857 } health_report { overall_health: HEALTHY } } }
Aug 07 15:11:58 dist-test-slave-42z9 krb5kdc[8938](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115517, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.194.62@KRBTEST.COM for oryx/127.29.194.62@KRBTEST.COM
I20260807 15:11:58.862516 9088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:58.855571 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56738 (local address 127.29.194.62:34237)
0807 15:11:58.855650 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:58.855653 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:58.855806 (+ 153us) server_negotiation.cc:407] Connection header received
0807 15:11:58.855880 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:58.855881 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:58.855891 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:58.855935 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:11:58.856467 (+ 532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:58.856988 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:58.857689 (+ 701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:58.857861 (+ 172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:58.857915 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:58.860201 (+ 2286us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:11:58.860202 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:11:58.860203 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:11:58.860210 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:11:58.861455 (+ 1245us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:58.861793 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:58.861794 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:58.861795 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:58.861846 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:58.862012 (+ 166us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:58.862013 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:58.862013 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:58.862122 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:11:58.862179 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0807 15:11:58.862321 (+ 142us) server_negotiation.cc:299] Negotiation successful
0807 15:11:58.862414 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260807 15:11:58.862591 9803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:58.855435 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:34237 (local address 127.0.0.1:56738)
0807 15:11:58.855712 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0807 15:11:58.855724 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 15:11:58.855820 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:11:58.855950 (+ 130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:11:58.855951 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:11:58.856320 (+ 369us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:11:58.856372 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:58.856374 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:58.857058 (+ 684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:58.857059 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:58.857622 (+ 563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:58.857624 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:58.857947 (+ 323us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:58.857948 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:58.857999 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:58.858617 (+ 618us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:11:58.858622 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:11:58.860120 (+ 1498us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:11:58.861525 (+ 1405us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:11:58.861533 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:11:58.861536 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:11:58.861728 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:11:58.861898 (+ 170us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:11:58.861899 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:11:58.861899 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:11:58.861963 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:11:58.862175 (+ 212us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:11:58.862184 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:11:58.862230 (+ 46us) client_negotiation.cc:769] Sending connection context
0807 15:11:58.862291 (+ 61us) client_negotiation.cc:240] Negotiation successful
0807 15:11:58.862478 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":243,"mutex_wait_us":77,"thread_start_us":164,"threads_started":1}
I20260807 15:11:58.864854 9008 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.29.194.62@KRBTEST.COM'} at 127.0.0.1:56738:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260807 15:11:58.865072 9008 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 15:11:58.866600 9214 tablet_service.cc:1467] Processing CreateTablet for tablet 22080e044de243ba931f77862d191c21 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=ea1d0fa078f34b8a8c40518f27c88456]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 15:11:58.866606 9410 tablet_service.cc:1467] Processing CreateTablet for tablet 22080e044de243ba931f77862d191c21 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=ea1d0fa078f34b8a8c40518f27c88456]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 15:11:58.866770 9214 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 22080e044de243ba931f77862d191c21. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:11:58.866775 9410 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 22080e044de243ba931f77862d191c21. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:11:58.867998 9759 tablet_bootstrap.cc:492] T 22080e044de243ba931f77862d191c21 P 7c8b57fd081d41bb8f74cf365d3e0a7a: Bootstrap starting.
I20260807 15:11:58.868120 9612 tablet_service.cc:1467] Processing CreateTablet for tablet 22080e044de243ba931f77862d191c21 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=ea1d0fa078f34b8a8c40518f27c88456]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 15:11:58.868234 9750 tablet_bootstrap.cc:492] T 22080e044de243ba931f77862d191c21 P 1c708c44654e461099098050b30d16f0: Bootstrap starting.
I20260807 15:11:58.868280 9612 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 22080e044de243ba931f77862d191c21. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:11:58.868453 9759 tablet_bootstrap.cc:654] T 22080e044de243ba931f77862d191c21 P 7c8b57fd081d41bb8f74cf365d3e0a7a: Neither blocks nor log segments found. Creating new log.
I20260807 15:11:58.868579 9750 tablet_bootstrap.cc:654] T 22080e044de243ba931f77862d191c21 P 1c708c44654e461099098050b30d16f0: Neither blocks nor log segments found. Creating new log.
I20260807 15:11:58.869259 9750 tablet_bootstrap.cc:492] T 22080e044de243ba931f77862d191c21 P 1c708c44654e461099098050b30d16f0: No bootstrap required, opened a new log
I20260807 15:11:58.869262 9759 tablet_bootstrap.cc:492] T 22080e044de243ba931f77862d191c21 P 7c8b57fd081d41bb8f74cf365d3e0a7a: No bootstrap required, opened a new log
I20260807 15:11:58.869311 9759 ts_tablet_manager.cc:1397] T 22080e044de243ba931f77862d191c21 P 7c8b57fd081d41bb8f74cf365d3e0a7a: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260807 15:11:58.869310 9750 ts_tablet_manager.cc:1397] T 22080e044de243ba931f77862d191c21 P 1c708c44654e461099098050b30d16f0: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260807 15:11:58.869468 9750 raft_consensus.cc:348] T 22080e044de243ba931f77862d191c21 P 1c708c44654e461099098050b30d16f0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c708c44654e461099098050b30d16f0" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 36541 } } peers { permanent_uuid: "7c8b57fd081d41bb8f74cf365d3e0a7a" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 36405 } } peers { permanent_uuid: "86b01224f5b34c19a27d23a31ab7306a" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 44857 } }
I20260807 15:11:58.869546 9750 raft_consensus.cc:374] T 22080e044de243ba931f77862d191c21 P 1c708c44654e461099098050b30d16f0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:11:58.869504 9759 raft_consensus.cc:348] T 22080e044de243ba931f77862d191c21 P 7c8b57fd081d41bb8f74cf365d3e0a7a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c708c44654e461099098050b30d16f0" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 36541 } } peers { permanent_uuid: "7c8b57fd081d41bb8f74cf365d3e0a7a" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 36405 } } peers { permanent_uuid: "86b01224f5b34c19a27d23a31ab7306a" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 44857 } }
I20260807 15:11:58.869565 9750 raft_consensus.cc:729] T 22080e044de243ba931f77862d191c21 P 1c708c44654e461099098050b30d16f0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1c708c44654e461099098050b30d16f0, State: Initialized, Role: FOLLOWER
I20260807 15:11:58.869577 9759 raft_consensus.cc:374] T 22080e044de243ba931f77862d191c21 P 7c8b57fd081d41bb8f74cf365d3e0a7a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:11:58.869597 9759 raft_consensus.cc:729] T 22080e044de243ba931f77862d191c21 P 7c8b57fd081d41bb8f74cf365d3e0a7a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c8b57fd081d41bb8f74cf365d3e0a7a, State: Initialized, Role: FOLLOWER
I20260807 15:11:58.869613 9750 consensus_queue.cc:260] T 22080e044de243ba931f77862d191c21 P 1c708c44654e461099098050b30d16f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c708c44654e461099098050b30d16f0" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 36541 } } peers { permanent_uuid: "7c8b57fd081d41bb8f74cf365d3e0a7a" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 36405 } } peers { permanent_uuid: "86b01224f5b34c19a27d23a31ab7306a" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 44857 } }
I20260807 15:11:58.869640 9759 consensus_queue.cc:260] T 22080e044de243ba931f77862d191c21 P 7c8b57fd081d41bb8f74cf365d3e0a7a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c708c44654e461099098050b30d16f0" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 36541 } } peers { permanent_uuid: "7c8b57fd081d41bb8f74cf365d3e0a7a" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 36405 } } peers { permanent_uuid: "86b01224f5b34c19a27d23a31ab7306a" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 44857 } }
I20260807 15:11:58.869706 9750 ts_tablet_manager.cc:1428] T 22080e044de243ba931f77862d191c21 P 1c708c44654e461099098050b30d16f0: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260807 15:11:58.869729 9759 ts_tablet_manager.cc:1428] T 22080e044de243ba931f77862d191c21 P 7c8b57fd081d41bb8f74cf365d3e0a7a: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260807 15:11:58.869832 9758 tablet_replica.cc:439] T 22080e044de243ba931f77862d191c21 P 1c708c44654e461099098050b30d16f0: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c708c44654e461099098050b30d16f0" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 36541 } } peers { permanent_uuid: "7c8b57fd081d41bb8f74cf365d3e0a7a" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 36405 } } peers { permanent_uuid: "86b01224f5b34c19a27d23a31ab7306a" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 44857 } } }
I20260807 15:11:58.869879 9761 tablet_replica.cc:439] T 22080e044de243ba931f77862d191c21 P 7c8b57fd081d41bb8f74cf365d3e0a7a: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c708c44654e461099098050b30d16f0" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 36541 } } peers { permanent_uuid: "7c8b57fd081d41bb8f74cf365d3e0a7a" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 36405 } } peers { permanent_uuid: "86b01224f5b34c19a27d23a31ab7306a" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 44857 } } }
I20260807 15:11:58.869920 9758 tablet_replica.cc:442] T 22080e044de243ba931f77862d191c21 P 1c708c44654e461099098050b30d16f0: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 15:11:58.869951 9761 tablet_replica.cc:442] T 22080e044de243ba931f77862d191c21 P 7c8b57fd081d41bb8f74cf365d3e0a7a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 15:11:58.870211 9749 tablet_bootstrap.cc:492] T 22080e044de243ba931f77862d191c21 P 86b01224f5b34c19a27d23a31ab7306a: Bootstrap starting.
I20260807 15:11:58.870694 9749 tablet_bootstrap.cc:654] T 22080e044de243ba931f77862d191c21 P 86b01224f5b34c19a27d23a31ab7306a: Neither blocks nor log segments found. Creating new log.
I20260807 15:11:58.871364 9749 tablet_bootstrap.cc:492] T 22080e044de243ba931f77862d191c21 P 86b01224f5b34c19a27d23a31ab7306a: No bootstrap required, opened a new log
I20260807 15:11:58.871414 9749 ts_tablet_manager.cc:1397] T 22080e044de243ba931f77862d191c21 P 86b01224f5b34c19a27d23a31ab7306a: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260807 15:11:58.871542 9749 raft_consensus.cc:348] T 22080e044de243ba931f77862d191c21 P 86b01224f5b34c19a27d23a31ab7306a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c708c44654e461099098050b30d16f0" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 36541 } } peers { permanent_uuid: "7c8b57fd081d41bb8f74cf365d3e0a7a" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 36405 } } peers { permanent_uuid: "86b01224f5b34c19a27d23a31ab7306a" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 44857 } }
I20260807 15:11:58.871610 9749 raft_consensus.cc:374] T 22080e044de243ba931f77862d191c21 P 86b01224f5b34c19a27d23a31ab7306a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:11:58.871624 9749 raft_consensus.cc:729] T 22080e044de243ba931f77862d191c21 P 86b01224f5b34c19a27d23a31ab7306a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 86b01224f5b34c19a27d23a31ab7306a, State: Initialized, Role: FOLLOWER
I20260807 15:11:58.871660 9749 consensus_queue.cc:260] T 22080e044de243ba931f77862d191c21 P 86b01224f5b34c19a27d23a31ab7306a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c708c44654e461099098050b30d16f0" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 36541 } } peers { permanent_uuid: "7c8b57fd081d41bb8f74cf365d3e0a7a" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 36405 } } peers { permanent_uuid: "86b01224f5b34c19a27d23a31ab7306a" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 44857 } }
I20260807 15:11:58.871753 9749 ts_tablet_manager.cc:1428] T 22080e044de243ba931f77862d191c21 P 86b01224f5b34c19a27d23a31ab7306a: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260807 15:11:58.871794 9755 tablet_replica.cc:439] T 22080e044de243ba931f77862d191c21 P 86b01224f5b34c19a27d23a31ab7306a: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c708c44654e461099098050b30d16f0" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 36541 } } peers { permanent_uuid: "7c8b57fd081d41bb8f74cf365d3e0a7a" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 36405 } } peers { permanent_uuid: "86b01224f5b34c19a27d23a31ab7306a" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 44857 } } }
I20260807 15:11:58.871888 9755 tablet_replica.cc:442] T 22080e044de243ba931f77862d191c21 P 86b01224f5b34c19a27d23a31ab7306a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 15:11:59.207713 9755 raft_consensus.cc:482] T 22080e044de243ba931f77862d191c21 P 86b01224f5b34c19a27d23a31ab7306a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 15:11:59.207827 9755 raft_consensus.cc:504] T 22080e044de243ba931f77862d191c21 P 86b01224f5b34c19a27d23a31ab7306a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c708c44654e461099098050b30d16f0" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 36541 } } peers { permanent_uuid: "7c8b57fd081d41bb8f74cf365d3e0a7a" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 36405 } } peers { permanent_uuid: "86b01224f5b34c19a27d23a31ab7306a" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 44857 } }
I20260807 15:11:59.208002 9755 leader_election.cc:288] T 22080e044de243ba931f77862d191c21 P 86b01224f5b34c19a27d23a31ab7306a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1c708c44654e461099098050b30d16f0 (127.29.194.1:36541), 7c8b57fd081d41bb8f74cf365d3e0a7a (127.29.194.2:36405)
I20260807 15:11:59.208273 9437 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "22080e044de243ba931f77862d191c21" candidate_uuid: "86b01224f5b34c19a27d23a31ab7306a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c8b57fd081d41bb8f74cf365d3e0a7a" is_pre_election: true
I20260807 15:11:59.208365 9437 raft_consensus.cc:2454] T 22080e044de243ba931f77862d191c21 P 7c8b57fd081d41bb8f74cf365d3e0a7a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 86b01224f5b34c19a27d23a31ab7306a in term 0.
I20260807 15:11:59.208297 9242 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "22080e044de243ba931f77862d191c21" candidate_uuid: "86b01224f5b34c19a27d23a31ab7306a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c708c44654e461099098050b30d16f0" is_pre_election: true
I20260807 15:11:59.208415 9242 raft_consensus.cc:2454] T 22080e044de243ba931f77862d191c21 P 1c708c44654e461099098050b30d16f0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 86b01224f5b34c19a27d23a31ab7306a in term 0.
I20260807 15:11:59.208532 9551 leader_election.cc:302] T 22080e044de243ba931f77862d191c21 P 86b01224f5b34c19a27d23a31ab7306a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c8b57fd081d41bb8f74cf365d3e0a7a, 86b01224f5b34c19a27d23a31ab7306a; no voters:
I20260807 15:11:59.208639 9755 raft_consensus.cc:2781] T 22080e044de243ba931f77862d191c21 P 86b01224f5b34c19a27d23a31ab7306a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 15:11:59.208688 9755 raft_consensus.cc:482] T 22080e044de243ba931f77862d191c21 P 86b01224f5b34c19a27d23a31ab7306a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 15:11:59.208710 9755 raft_consensus.cc:3037] T 22080e044de243ba931f77862d191c21 P 86b01224f5b34c19a27d23a31ab7306a [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:11:59.209426 9755 raft_consensus.cc:504] T 22080e044de243ba931f77862d191c21 P 86b01224f5b34c19a27d23a31ab7306a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c708c44654e461099098050b30d16f0" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 36541 } } peers { permanent_uuid: "7c8b57fd081d41bb8f74cf365d3e0a7a" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 36405 } } peers { permanent_uuid: "86b01224f5b34c19a27d23a31ab7306a" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 44857 } }
I20260807 15:11:59.209622 9755 leader_election.cc:288] T 22080e044de243ba931f77862d191c21 P 86b01224f5b34c19a27d23a31ab7306a [CANDIDATE]: Term 1 election: Requested vote from peers 1c708c44654e461099098050b30d16f0 (127.29.194.1:36541), 7c8b57fd081d41bb8f74cf365d3e0a7a (127.29.194.2:36405)
I20260807 15:11:59.209785 9242 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "22080e044de243ba931f77862d191c21" candidate_uuid: "86b01224f5b34c19a27d23a31ab7306a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c708c44654e461099098050b30d16f0"
I20260807 15:11:59.209800 9437 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "22080e044de243ba931f77862d191c21" candidate_uuid: "86b01224f5b34c19a27d23a31ab7306a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c8b57fd081d41bb8f74cf365d3e0a7a"
I20260807 15:11:59.209875 9437 raft_consensus.cc:3037] T 22080e044de243ba931f77862d191c21 P 7c8b57fd081d41bb8f74cf365d3e0a7a [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:11:59.209875 9242 raft_consensus.cc:3037] T 22080e044de243ba931f77862d191c21 P 1c708c44654e461099098050b30d16f0 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:11:59.210678 9242 raft_consensus.cc:2454] T 22080e044de243ba931f77862d191c21 P 1c708c44654e461099098050b30d16f0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 86b01224f5b34c19a27d23a31ab7306a in term 1.
I20260807 15:11:59.210678 9437 raft_consensus.cc:2454] T 22080e044de243ba931f77862d191c21 P 7c8b57fd081d41bb8f74cf365d3e0a7a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 86b01224f5b34c19a27d23a31ab7306a in term 1.
I20260807 15:11:59.210899 9551 leader_election.cc:302] T 22080e044de243ba931f77862d191c21 P 86b01224f5b34c19a27d23a31ab7306a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c8b57fd081d41bb8f74cf365d3e0a7a, 86b01224f5b34c19a27d23a31ab7306a; no voters:
I20260807 15:11:59.211012 9755 raft_consensus.cc:2781] T 22080e044de243ba931f77862d191c21 P 86b01224f5b34c19a27d23a31ab7306a [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:11:59.211066 9755 raft_consensus.cc:686] T 22080e044de243ba931f77862d191c21 P 86b01224f5b34c19a27d23a31ab7306a [term 1 LEADER]: Becoming Leader. State: Replica: 86b01224f5b34c19a27d23a31ab7306a, State: Running, Role: LEADER
I20260807 15:11:59.211117 9755 consensus_queue.cc:237] T 22080e044de243ba931f77862d191c21 P 86b01224f5b34c19a27d23a31ab7306a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c708c44654e461099098050b30d16f0" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 36541 } } peers { permanent_uuid: "7c8b57fd081d41bb8f74cf365d3e0a7a" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 36405 } } peers { permanent_uuid: "86b01224f5b34c19a27d23a31ab7306a" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 44857 } }
I20260807 15:11:59.211408 9792 tablet_replica.cc:439] T 22080e044de243ba931f77862d191c21 P 86b01224f5b34c19a27d23a31ab7306a: TxnStatusTablet state changed. Reason: New leader 86b01224f5b34c19a27d23a31ab7306a. Latest consensus state: current_term: 1 leader_uuid: "86b01224f5b34c19a27d23a31ab7306a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c708c44654e461099098050b30d16f0" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 36541 } } peers { permanent_uuid: "7c8b57fd081d41bb8f74cf365d3e0a7a" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 36405 } } peers { permanent_uuid: "86b01224f5b34c19a27d23a31ab7306a" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 44857 } } }
I20260807 15:11:59.211488 9792 tablet_replica.cc:442] T 22080e044de243ba931f77862d191c21 P 86b01224f5b34c19a27d23a31ab7306a: This TxnStatusTablet replica's current role is: LEADER
I20260807 15:11:59.211623 9811 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260807 15:11:59.211762 9811 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260807 15:11:59.211903 9008 catalog_manager.cc:5495] T 22080e044de243ba931f77862d191c21 P 86b01224f5b34c19a27d23a31ab7306a reported cstate change: term changed from 0 to 1, leader changed from <none> to 86b01224f5b34c19a27d23a31ab7306a (127.29.194.3). New cstate: current_term: 1 leader_uuid: "86b01224f5b34c19a27d23a31ab7306a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c708c44654e461099098050b30d16f0" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 36541 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7c8b57fd081d41bb8f74cf365d3e0a7a" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 36405 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "86b01224f5b34c19a27d23a31ab7306a" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 44857 } health_report { overall_health: HEALTHY } } }
I20260807 15:11:59.229342 9803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:59.225859 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.3:44857 (local address 127.0.0.1:46376)
0807 15:11:59.225973 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0807 15:11:59.225985 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 15:11:59.226035 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:11:59.226409 (+ 374us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:11:59.226410 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:11:59.226413 (+ 3us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 15:11:59.226473 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:59.226476 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:59.227522 (+ 1046us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:59.227523 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:59.228155 (+ 632us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:59.228157 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:59.228684 (+ 527us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:59.228686 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:59.228737 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:59.228764 (+ 27us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 15:11:59.229179 (+ 415us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 15:11:59.229188 (+ 9us) client_negotiation.cc:769] Sending connection context
0807 15:11:59.229220 (+ 32us) client_negotiation.cc:240] Negotiation successful
0807 15:11:59.229271 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":57}
I20260807 15:11:59.229391 9814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:59.225978 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46376 (local address 127.29.194.3:44857)
0807 15:11:59.226125 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:59.226127 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:59.226140 (+ 13us) server_negotiation.cc:407] Connection header received
0807 15:11:59.226192 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:59.226193 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:59.226218 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:59.226342 (+ 124us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 15:11:59.226571 (+ 229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:59.227392 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:59.228249 (+ 857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:59.228578 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:59.228653 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:59.228905 (+ 252us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 15:11:59.229100 (+ 195us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 15:11:59.229191 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0807 15:11:59.229239 (+ 48us) server_negotiation.cc:299] Negotiation successful
0807 15:11:59.229265 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123,"thread_start_us":83,"threads_started":1}
I20260807 15:11:59.231032 9437 raft_consensus.cc:1261] T 22080e044de243ba931f77862d191c21 P 7c8b57fd081d41bb8f74cf365d3e0a7a [term 1 FOLLOWER]: Refusing update from remote peer 86b01224f5b34c19a27d23a31ab7306a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:11:59.231084 9242 raft_consensus.cc:1261] T 22080e044de243ba931f77862d191c21 P 1c708c44654e461099098050b30d16f0 [term 1 FOLLOWER]: Refusing update from remote peer 86b01224f5b34c19a27d23a31ab7306a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:11:59.231355 9755 consensus_queue.cc:1035] T 22080e044de243ba931f77862d191c21 P 86b01224f5b34c19a27d23a31ab7306a [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c8b57fd081d41bb8f74cf365d3e0a7a" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 36405 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:11:59.231504 9792 consensus_queue.cc:1035] T 22080e044de243ba931f77862d191c21 P 86b01224f5b34c19a27d23a31ab7306a [LEADER]: Connected to new peer: Peer: permanent_uuid: "1c708c44654e461099098050b30d16f0" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 36541 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:11:59.232118 9761 tablet_replica.cc:439] T 22080e044de243ba931f77862d191c21 P 7c8b57fd081d41bb8f74cf365d3e0a7a: TxnStatusTablet state changed. Reason: New leader 86b01224f5b34c19a27d23a31ab7306a. Latest consensus state: current_term: 1 leader_uuid: "86b01224f5b34c19a27d23a31ab7306a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c708c44654e461099098050b30d16f0" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 36541 } } peers { permanent_uuid: "7c8b57fd081d41bb8f74cf365d3e0a7a" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 36405 } } peers { permanent_uuid: "86b01224f5b34c19a27d23a31ab7306a" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 44857 } } }
I20260807 15:11:59.232214 9761 tablet_replica.cc:442] T 22080e044de243ba931f77862d191c21 P 7c8b57fd081d41bb8f74cf365d3e0a7a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 15:11:59.232237 9758 tablet_replica.cc:439] T 22080e044de243ba931f77862d191c21 P 1c708c44654e461099098050b30d16f0: TxnStatusTablet state changed. Reason: New leader 86b01224f5b34c19a27d23a31ab7306a. Latest consensus state: current_term: 1 leader_uuid: "86b01224f5b34c19a27d23a31ab7306a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c708c44654e461099098050b30d16f0" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 36541 } } peers { permanent_uuid: "7c8b57fd081d41bb8f74cf365d3e0a7a" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 36405 } } peers { permanent_uuid: "86b01224f5b34c19a27d23a31ab7306a" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 44857 } } }
I20260807 15:11:59.232327 9758 tablet_replica.cc:442] T 22080e044de243ba931f77862d191c21 P 1c708c44654e461099098050b30d16f0: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 15:11:59.233043 9755 tablet_replica.cc:439] T 22080e044de243ba931f77862d191c21 P 86b01224f5b34c19a27d23a31ab7306a: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "86b01224f5b34c19a27d23a31ab7306a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c708c44654e461099098050b30d16f0" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 36541 } } peers { permanent_uuid: "7c8b57fd081d41bb8f74cf365d3e0a7a" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 36405 } } peers { permanent_uuid: "86b01224f5b34c19a27d23a31ab7306a" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 44857 } } }
I20260807 15:11:59.233129 9755 tablet_replica.cc:442] T 22080e044de243ba931f77862d191c21 P 86b01224f5b34c19a27d23a31ab7306a: This TxnStatusTablet replica's current role is: LEADER
I20260807 15:11:59.233193 9815 mvcc.cc:205] Tried to move back new op lower bound from 7315929166768001024 to 7315929166689005568. Current Snapshot: MvccSnapshot[applied={T|T < 7315929166768001024 or (T in {7315929166768001024})}]
I20260807 15:11:59.233872 9758 tablet_replica.cc:439] T 22080e044de243ba931f77862d191c21 P 1c708c44654e461099098050b30d16f0: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "86b01224f5b34c19a27d23a31ab7306a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c708c44654e461099098050b30d16f0" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 36541 } } peers { permanent_uuid: "7c8b57fd081d41bb8f74cf365d3e0a7a" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 36405 } } peers { permanent_uuid: "86b01224f5b34c19a27d23a31ab7306a" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 44857 } } }
I20260807 15:11:59.233958 9758 tablet_replica.cc:442] T 22080e044de243ba931f77862d191c21 P 1c708c44654e461099098050b30d16f0: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 15:11:59.234814 9755 tablet_replica.cc:439] T 22080e044de243ba931f77862d191c21 P 86b01224f5b34c19a27d23a31ab7306a: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "86b01224f5b34c19a27d23a31ab7306a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c708c44654e461099098050b30d16f0" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 36541 } } peers { permanent_uuid: "7c8b57fd081d41bb8f74cf365d3e0a7a" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 36405 } } peers { permanent_uuid: "86b01224f5b34c19a27d23a31ab7306a" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 44857 } } }
I20260807 15:11:59.234987 9755 tablet_replica.cc:442] T 22080e044de243ba931f77862d191c21 P 86b01224f5b34c19a27d23a31ab7306a: This TxnStatusTablet replica's current role is: LEADER
I20260807 15:11:59.235133 9761 tablet_replica.cc:439] T 22080e044de243ba931f77862d191c21 P 7c8b57fd081d41bb8f74cf365d3e0a7a: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "86b01224f5b34c19a27d23a31ab7306a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c708c44654e461099098050b30d16f0" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 36541 } } peers { permanent_uuid: "7c8b57fd081d41bb8f74cf365d3e0a7a" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 36405 } } peers { permanent_uuid: "86b01224f5b34c19a27d23a31ab7306a" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 44857 } } }
I20260807 15:11:59.235225 9761 tablet_replica.cc:442] T 22080e044de243ba931f77862d191c21 P 7c8b57fd081d41bb8f74cf365d3e0a7a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 15:11:59.238616 9814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:59.235246 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46382 (local address 127.29.194.3:44857)
0807 15:11:59.235488 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:59.235491 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:59.235499 (+ 8us) server_negotiation.cc:407] Connection header received
0807 15:11:59.235604 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:59.235604 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:59.235615 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:59.235846 (+ 231us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 15:11:59.235876 (+ 30us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:59.236659 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:59.237620 (+ 961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:59.237928 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:59.238015 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:59.238196 (+ 181us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 15:11:59.238234 (+ 38us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 15:11:59.238288 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0807 15:11:59.238497 (+ 209us) server_negotiation.cc:299] Negotiation successful
0807 15:11:59.238536 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":222}
I20260807 15:11:59.242100 9792 consensus_queue.cc:1035] T 91f6d1ab0a5b4a73a55e3f82d309e53e P 86b01224f5b34c19a27d23a31ab7306a [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c8b57fd081d41bb8f74cf365d3e0a7a" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 36405 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:11:59.244675 9755 consensus_queue.cc:1035] T 91f6d1ab0a5b4a73a55e3f82d309e53e P 86b01224f5b34c19a27d23a31ab7306a [LEADER]: Connected to new peer: Peer: permanent_uuid: "1c708c44654e461099098050b30d16f0" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 36541 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:11:59.247290 9814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:59.243321 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.194.3:56333 (local address 127.29.194.3:44857)
0807 15:11:59.244154 (+ 833us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:59.244157 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:59.244530 (+ 373us) server_negotiation.cc:407] Connection header received
0807 15:11:59.244630 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:59.244631 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:59.244640 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:59.244666 (+ 26us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 15:11:59.244873 (+ 207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:59.245513 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:59.246543 (+ 1030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:59.246781 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:59.246822 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:59.247052 (+ 230us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 15:11:59.247084 (+ 32us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 15:11:59.247109 (+ 25us) server_negotiation.cc:1035] Waiting for connection context
0807 15:11:59.247192 (+ 83us) server_negotiation.cc:299] Negotiation successful
0807 15:11:59.247217 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":804}
I20260807 15:11:59.247740 9831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:59.242920 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.3:44857 (local address 127.29.194.3:56333)
0807 15:11:59.244410 (+ 1490us) negotiation.cc:107] Waiting for socket to connect
0807 15:11:59.244432 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0807 15:11:59.244565 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:11:59.244720 (+ 155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:11:59.244721 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:11:59.244722 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 15:11:59.244806 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:59.244808 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:59.245594 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:59.245594 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:59.246455 (+ 861us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:59.246459 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:59.246869 (+ 410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:59.246870 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:59.246938 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:59.246962 (+ 24us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 15:11:59.247141 (+ 179us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 15:11:59.247142 (+ 1us) client_negotiation.cc:769] Sending connection context
0807 15:11:59.247428 (+ 286us) client_negotiation.cc:240] Negotiation successful
0807 15:11:59.247472 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1280,"thread_start_us":87,"threads_started":1}
I20260807 15:11:59.263630 9009 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56732:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 15:11:59.263767 9009 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56732:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 15:11:59.264432 9009 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f743aff5796d4ca99c67bbf7b9b722d6: Sending DeleteTablet for 3 replicas of tablet 91f6d1ab0a5b4a73a55e3f82d309e53e
I20260807 15:11:59.264760 9214 tablet_service.cc:1514] Processing DeleteTablet for tablet 91f6d1ab0a5b4a73a55e3f82d309e53e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:11:59 UTC) from {username='oryx', principal='oryx/127.29.194.62@KRBTEST.COM'} at 127.0.0.1:34040
I20260807 15:11:59.264824 9410 tablet_service.cc:1514] Processing DeleteTablet for tablet 91f6d1ab0a5b4a73a55e3f82d309e53e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:11:59 UTC) from {username='oryx', principal='oryx/127.29.194.62@KRBTEST.COM'} at 127.0.0.1:35692
I20260807 15:11:59.265043 9612 tablet_service.cc:1514] Processing DeleteTablet for tablet 91f6d1ab0a5b4a73a55e3f82d309e53e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:11:59 UTC) from {username='oryx', principal='oryx/127.29.194.62@KRBTEST.COM'} at 127.0.0.1:46368
I20260807 15:11:59.265547 9838 tablet_replica.cc:330] T 91f6d1ab0a5b4a73a55e3f82d309e53e P 1c708c44654e461099098050b30d16f0: stopping tablet replica
I20260807 15:11:59.265609 9838 raft_consensus.cc:2229] T 91f6d1ab0a5b4a73a55e3f82d309e53e P 1c708c44654e461099098050b30d16f0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 15:11:59.265645 9838 raft_consensus.cc:2258] T 91f6d1ab0a5b4a73a55e3f82d309e53e P 1c708c44654e461099098050b30d16f0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:11:59.265928 9838 ts_tablet_manager.cc:1905] T 91f6d1ab0a5b4a73a55e3f82d309e53e P 1c708c44654e461099098050b30d16f0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:11:59.267727 9838 ts_tablet_manager.cc:1918] T 91f6d1ab0a5b4a73a55e3f82d309e53e P 1c708c44654e461099098050b30d16f0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 15:11:59.267798 9838 log.cc:1196] T 91f6d1ab0a5b4a73a55e3f82d309e53e P 1c708c44654e461099098050b30d16f0: Deleting WAL directory at /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115480851507-30472-0/minicluster-data/ts-0/wal/wals/91f6d1ab0a5b4a73a55e3f82d309e53e
I20260807 15:11:59.268060 9838 ts_tablet_manager.cc:1939] T 91f6d1ab0a5b4a73a55e3f82d309e53e P 1c708c44654e461099098050b30d16f0: Deleting consensus metadata
I20260807 15:11:59.268246 9839 tablet_replica.cc:330] T 91f6d1ab0a5b4a73a55e3f82d309e53e P 7c8b57fd081d41bb8f74cf365d3e0a7a: stopping tablet replica
I20260807 15:11:59.268314 9839 raft_consensus.cc:2229] T 91f6d1ab0a5b4a73a55e3f82d309e53e P 7c8b57fd081d41bb8f74cf365d3e0a7a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 15:11:59.268375 9839 raft_consensus.cc:2258] T 91f6d1ab0a5b4a73a55e3f82d309e53e P 7c8b57fd081d41bb8f74cf365d3e0a7a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:11:59.268651 9843 tablet_replica.cc:330] T 91f6d1ab0a5b4a73a55e3f82d309e53e P 86b01224f5b34c19a27d23a31ab7306a: stopping tablet replica
I20260807 15:11:59.268718 9843 raft_consensus.cc:2229] T 91f6d1ab0a5b4a73a55e3f82d309e53e P 86b01224f5b34c19a27d23a31ab7306a [term 1 LEADER]: Raft consensus shutting down.
I20260807 15:11:59.268713 8985 catalog_manager.cc:4841] TS 1c708c44654e461099098050b30d16f0 (127.29.194.1:36541): tablet 91f6d1ab0a5b4a73a55e3f82d309e53e (table test-table [id=0a2c77d43c7e4d46b2bd3a64fbb92336]) successfully deleted
I20260807 15:11:59.268829 9843 raft_consensus.cc:2258] T 91f6d1ab0a5b4a73a55e3f82d309e53e P 86b01224f5b34c19a27d23a31ab7306a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:11:59.269033 9839 ts_tablet_manager.cc:1905] T 91f6d1ab0a5b4a73a55e3f82d309e53e P 7c8b57fd081d41bb8f74cf365d3e0a7a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:11:59.269842 9843 ts_tablet_manager.cc:1905] T 91f6d1ab0a5b4a73a55e3f82d309e53e P 86b01224f5b34c19a27d23a31ab7306a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:11:59.270643 9839 ts_tablet_manager.cc:1918] T 91f6d1ab0a5b4a73a55e3f82d309e53e P 7c8b57fd081d41bb8f74cf365d3e0a7a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 15:11:59.270721 9839 log.cc:1196] T 91f6d1ab0a5b4a73a55e3f82d309e53e P 7c8b57fd081d41bb8f74cf365d3e0a7a: Deleting WAL directory at /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115480851507-30472-0/minicluster-data/ts-1/wal/wals/91f6d1ab0a5b4a73a55e3f82d309e53e
I20260807 15:11:59.270927 9839 ts_tablet_manager.cc:1939] T 91f6d1ab0a5b4a73a55e3f82d309e53e P 7c8b57fd081d41bb8f74cf365d3e0a7a: Deleting consensus metadata
I20260807 15:11:59.271478 8986 catalog_manager.cc:4841] TS 7c8b57fd081d41bb8f74cf365d3e0a7a (127.29.194.2:36405): tablet 91f6d1ab0a5b4a73a55e3f82d309e53e (table test-table [id=0a2c77d43c7e4d46b2bd3a64fbb92336]) successfully deleted
I20260807 15:11:59.272001 9843 ts_tablet_manager.cc:1918] T 91f6d1ab0a5b4a73a55e3f82d309e53e P 86b01224f5b34c19a27d23a31ab7306a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 15:11:59.272070 9843 log.cc:1196] T 91f6d1ab0a5b4a73a55e3f82d309e53e P 86b01224f5b34c19a27d23a31ab7306a: Deleting WAL directory at /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115480851507-30472-0/minicluster-data/ts-2/wal/wals/91f6d1ab0a5b4a73a55e3f82d309e53e
I20260807 15:11:59.272262 9843 ts_tablet_manager.cc:1939] T 91f6d1ab0a5b4a73a55e3f82d309e53e P 86b01224f5b34c19a27d23a31ab7306a: Deleting consensus metadata
I20260807 15:11:59.272841 8985 catalog_manager.cc:4841] TS 86b01224f5b34c19a27d23a31ab7306a (127.29.194.3:44857): tablet 91f6d1ab0a5b4a73a55e3f82d309e53e (table test-table [id=0a2c77d43c7e4d46b2bd3a64fbb92336]) successfully deleted
Aug 07 15:11:59 dist-test-slave-42z9 krb5kdc[8938](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115519, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 07 15:11:59 dist-test-slave-42z9 krb5kdc[8938](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.29.194.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 15:11:59 dist-test-slave-42z9 krb5kdc[8938](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.29.194.62@KRBTEST.COM, Server not found in Kerberos database
W20260807 15:11:59.294975 9088 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:11:59.288720 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56746 (local address 127.29.194.62:34237)
0807 15:11:59.288779 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:59.288781 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:59.288971 (+ 190us) server_negotiation.cc:407] Connection header received
0807 15:11:59.289051 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:59.289052 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:59.289062 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:59.289112 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:11:59.290076 (+ 964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:59.290843 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:59.291729 (+ 886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:59.291918 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:59.291976 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:59.294895 (+ 2919us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56746: BlockingRecv error: recv got EOF from 127.0.0.1:56746 (error 108)
Metrics: {"server-negotiator.queue_time_us":29}
Aug 07 15:11:59 dist-test-slave-42z9 krb5kdc[8938](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115519, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.29.194.62@KRBTEST.COM
I20260807 15:11:59.302978 9088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:59.296055 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56754 (local address 127.29.194.62:34237)
0807 15:11:59.296112 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:59.296114 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:59.296121 (+ 7us) server_negotiation.cc:407] Connection header received
0807 15:11:59.296144 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:59.296145 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:59.296154 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:59.296197 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:11:59.296667 (+ 470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:59.297169 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:59.297901 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:59.298107 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:59.298162 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:59.300570 (+ 2408us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:11:59.300572 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:11:59.300572 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:11:59.300581 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:11:59.301956 (+ 1375us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:59.302272 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:59.302273 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:59.302274 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:59.302326 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:59.302484 (+ 158us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:59.302485 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:59.302485 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:59.302601 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:11:59.302689 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0807 15:11:59.302801 (+ 112us) server_negotiation.cc:299] Negotiation successful
0807 15:11:59.302881 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260807 15:11:59.304579 9009 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56754:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 15:11:59.304734 9009 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 15:11:59.306326 9214 tablet_service.cc:1467] Processing CreateTablet for tablet 9e534f05468e48dca3301b0d578bbf05 (DEFAULT_TABLE table=test-table [id=efc700fe31b74a6cbf6217fc8d242c4c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:11:59.306326 9410 tablet_service.cc:1467] Processing CreateTablet for tablet 9e534f05468e48dca3301b0d578bbf05 (DEFAULT_TABLE table=test-table [id=efc700fe31b74a6cbf6217fc8d242c4c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:11:59.306495 9214 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9e534f05468e48dca3301b0d578bbf05. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:11:59.306495 9410 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9e534f05468e48dca3301b0d578bbf05. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:11:59.307879 9759 tablet_bootstrap.cc:492] T 9e534f05468e48dca3301b0d578bbf05 P 7c8b57fd081d41bb8f74cf365d3e0a7a: Bootstrap starting.
I20260807 15:11:59.307914 9612 tablet_service.cc:1467] Processing CreateTablet for tablet 9e534f05468e48dca3301b0d578bbf05 (DEFAULT_TABLE table=test-table [id=efc700fe31b74a6cbf6217fc8d242c4c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:11:59.308058 9612 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9e534f05468e48dca3301b0d578bbf05. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:11:59.308343 9759 tablet_bootstrap.cc:654] T 9e534f05468e48dca3301b0d578bbf05 P 7c8b57fd081d41bb8f74cf365d3e0a7a: Neither blocks nor log segments found. Creating new log.
I20260807 15:11:59.308703 9750 tablet_bootstrap.cc:492] T 9e534f05468e48dca3301b0d578bbf05 P 1c708c44654e461099098050b30d16f0: Bootstrap starting.
I20260807 15:11:59.309105 9750 tablet_bootstrap.cc:654] T 9e534f05468e48dca3301b0d578bbf05 P 1c708c44654e461099098050b30d16f0: Neither blocks nor log segments found. Creating new log.
I20260807 15:11:59.309122 9749 tablet_bootstrap.cc:492] T 9e534f05468e48dca3301b0d578bbf05 P 86b01224f5b34c19a27d23a31ab7306a: Bootstrap starting.
I20260807 15:11:59.309542 9749 tablet_bootstrap.cc:654] T 9e534f05468e48dca3301b0d578bbf05 P 86b01224f5b34c19a27d23a31ab7306a: Neither blocks nor log segments found. Creating new log.
I20260807 15:11:59.309871 9759 tablet_bootstrap.cc:492] T 9e534f05468e48dca3301b0d578bbf05 P 7c8b57fd081d41bb8f74cf365d3e0a7a: No bootstrap required, opened a new log
I20260807 15:11:59.309934 9759 ts_tablet_manager.cc:1397] T 9e534f05468e48dca3301b0d578bbf05 P 7c8b57fd081d41bb8f74cf365d3e0a7a: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 15:11:59.310189 9749 tablet_bootstrap.cc:492] T 9e534f05468e48dca3301b0d578bbf05 P 86b01224f5b34c19a27d23a31ab7306a: No bootstrap required, opened a new log
I20260807 15:11:59.310143 9759 raft_consensus.cc:348] T 9e534f05468e48dca3301b0d578bbf05 P 7c8b57fd081d41bb8f74cf365d3e0a7a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c708c44654e461099098050b30d16f0" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 36541 } } peers { permanent_uuid: "7c8b57fd081d41bb8f74cf365d3e0a7a" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 36405 } } peers { permanent_uuid: "86b01224f5b34c19a27d23a31ab7306a" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 44857 } }
I20260807 15:11:59.310235 9749 ts_tablet_manager.cc:1397] T 9e534f05468e48dca3301b0d578bbf05 P 86b01224f5b34c19a27d23a31ab7306a: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260807 15:11:59.310254 9759 raft_consensus.cc:374] T 9e534f05468e48dca3301b0d578bbf05 P 7c8b57fd081d41bb8f74cf365d3e0a7a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:11:59.310281 9759 raft_consensus.cc:729] T 9e534f05468e48dca3301b0d578bbf05 P 7c8b57fd081d41bb8f74cf365d3e0a7a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c8b57fd081d41bb8f74cf365d3e0a7a, State: Initialized, Role: FOLLOWER
I20260807 15:11:59.310333 9759 consensus_queue.cc:260] T 9e534f05468e48dca3301b0d578bbf05 P 7c8b57fd081d41bb8f74cf365d3e0a7a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c708c44654e461099098050b30d16f0" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 36541 } } peers { permanent_uuid: "7c8b57fd081d41bb8f74cf365d3e0a7a" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 36405 } } peers { permanent_uuid: "86b01224f5b34c19a27d23a31ab7306a" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 44857 } }
I20260807 15:11:59.310366 9749 raft_consensus.cc:348] T 9e534f05468e48dca3301b0d578bbf05 P 86b01224f5b34c19a27d23a31ab7306a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c708c44654e461099098050b30d16f0" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 36541 } } peers { permanent_uuid: "7c8b57fd081d41bb8f74cf365d3e0a7a" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 36405 } } peers { permanent_uuid: "86b01224f5b34c19a27d23a31ab7306a" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 44857 } }
I20260807 15:11:59.310432 9749 raft_consensus.cc:374] T 9e534f05468e48dca3301b0d578bbf05 P 86b01224f5b34c19a27d23a31ab7306a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:11:59.310428 9759 ts_tablet_manager.cc:1428] T 9e534f05468e48dca3301b0d578bbf05 P 7c8b57fd081d41bb8f74cf365d3e0a7a: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260807 15:11:59.310451 9749 raft_consensus.cc:729] T 9e534f05468e48dca3301b0d578bbf05 P 86b01224f5b34c19a27d23a31ab7306a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 86b01224f5b34c19a27d23a31ab7306a, State: Initialized, Role: FOLLOWER
I20260807 15:11:59.310485 9749 consensus_queue.cc:260] T 9e534f05468e48dca3301b0d578bbf05 P 86b01224f5b34c19a27d23a31ab7306a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c708c44654e461099098050b30d16f0" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 36541 } } peers { permanent_uuid: "7c8b57fd081d41bb8f74cf365d3e0a7a" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 36405 } } peers { permanent_uuid: "86b01224f5b34c19a27d23a31ab7306a" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 44857 } }
I20260807 15:11:59.310577 9749 ts_tablet_manager.cc:1428] T 9e534f05468e48dca3301b0d578bbf05 P 86b01224f5b34c19a27d23a31ab7306a: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260807 15:11:59.311000 9750 tablet_bootstrap.cc:492] T 9e534f05468e48dca3301b0d578bbf05 P 1c708c44654e461099098050b30d16f0: No bootstrap required, opened a new log
I20260807 15:11:59.311048 9750 ts_tablet_manager.cc:1397] T 9e534f05468e48dca3301b0d578bbf05 P 1c708c44654e461099098050b30d16f0: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260807 15:11:59.311156 9750 raft_consensus.cc:348] T 9e534f05468e48dca3301b0d578bbf05 P 1c708c44654e461099098050b30d16f0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c708c44654e461099098050b30d16f0" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 36541 } } peers { permanent_uuid: "7c8b57fd081d41bb8f74cf365d3e0a7a" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 36405 } } peers { permanent_uuid: "86b01224f5b34c19a27d23a31ab7306a" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 44857 } }
I20260807 15:11:59.311240 9750 raft_consensus.cc:374] T 9e534f05468e48dca3301b0d578bbf05 P 1c708c44654e461099098050b30d16f0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:11:59.311268 9750 raft_consensus.cc:729] T 9e534f05468e48dca3301b0d578bbf05 P 1c708c44654e461099098050b30d16f0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1c708c44654e461099098050b30d16f0, State: Initialized, Role: FOLLOWER
I20260807 15:11:59.311309 9750 consensus_queue.cc:260] T 9e534f05468e48dca3301b0d578bbf05 P 1c708c44654e461099098050b30d16f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c708c44654e461099098050b30d16f0" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 36541 } } peers { permanent_uuid: "7c8b57fd081d41bb8f74cf365d3e0a7a" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 36405 } } peers { permanent_uuid: "86b01224f5b34c19a27d23a31ab7306a" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 44857 } }
I20260807 15:11:59.311403 9750 ts_tablet_manager.cc:1428] T 9e534f05468e48dca3301b0d578bbf05 P 1c708c44654e461099098050b30d16f0: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260807 15:11:59.453181 9836 raft_consensus.cc:482] T 9e534f05468e48dca3301b0d578bbf05 P 86b01224f5b34c19a27d23a31ab7306a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 15:11:59.453298 9836 raft_consensus.cc:504] T 9e534f05468e48dca3301b0d578bbf05 P 86b01224f5b34c19a27d23a31ab7306a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c708c44654e461099098050b30d16f0" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 36541 } } peers { permanent_uuid: "7c8b57fd081d41bb8f74cf365d3e0a7a" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 36405 } } peers { permanent_uuid: "86b01224f5b34c19a27d23a31ab7306a" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 44857 } }
I20260807 15:11:59.453564 9836 leader_election.cc:288] T 9e534f05468e48dca3301b0d578bbf05 P 86b01224f5b34c19a27d23a31ab7306a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1c708c44654e461099098050b30d16f0 (127.29.194.1:36541), 7c8b57fd081d41bb8f74cf365d3e0a7a (127.29.194.2:36405)
I20260807 15:11:59.453804 9437 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9e534f05468e48dca3301b0d578bbf05" candidate_uuid: "86b01224f5b34c19a27d23a31ab7306a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c8b57fd081d41bb8f74cf365d3e0a7a" is_pre_election: true
I20260807 15:11:59.453907 9437 raft_consensus.cc:2454] T 9e534f05468e48dca3301b0d578bbf05 P 7c8b57fd081d41bb8f74cf365d3e0a7a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 86b01224f5b34c19a27d23a31ab7306a in term 0.
I20260807 15:11:59.453850 9241 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9e534f05468e48dca3301b0d578bbf05" candidate_uuid: "86b01224f5b34c19a27d23a31ab7306a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c708c44654e461099098050b30d16f0" is_pre_election: true
I20260807 15:11:59.453976 9241 raft_consensus.cc:2454] T 9e534f05468e48dca3301b0d578bbf05 P 1c708c44654e461099098050b30d16f0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 86b01224f5b34c19a27d23a31ab7306a in term 0.
I20260807 15:11:59.454087 9551 leader_election.cc:302] T 9e534f05468e48dca3301b0d578bbf05 P 86b01224f5b34c19a27d23a31ab7306a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c8b57fd081d41bb8f74cf365d3e0a7a, 86b01224f5b34c19a27d23a31ab7306a; no voters:
I20260807 15:11:59.454171 9836 raft_consensus.cc:2781] T 9e534f05468e48dca3301b0d578bbf05 P 86b01224f5b34c19a27d23a31ab7306a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 15:11:59.454207 9836 raft_consensus.cc:482] T 9e534f05468e48dca3301b0d578bbf05 P 86b01224f5b34c19a27d23a31ab7306a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 15:11:59.454233 9836 raft_consensus.cc:3037] T 9e534f05468e48dca3301b0d578bbf05 P 86b01224f5b34c19a27d23a31ab7306a [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:11:59.455118 9836 raft_consensus.cc:504] T 9e534f05468e48dca3301b0d578bbf05 P 86b01224f5b34c19a27d23a31ab7306a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c708c44654e461099098050b30d16f0" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 36541 } } peers { permanent_uuid: "7c8b57fd081d41bb8f74cf365d3e0a7a" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 36405 } } peers { permanent_uuid: "86b01224f5b34c19a27d23a31ab7306a" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 44857 } }
I20260807 15:11:59.455260 9836 leader_election.cc:288] T 9e534f05468e48dca3301b0d578bbf05 P 86b01224f5b34c19a27d23a31ab7306a [CANDIDATE]: Term 1 election: Requested vote from peers 1c708c44654e461099098050b30d16f0 (127.29.194.1:36541), 7c8b57fd081d41bb8f74cf365d3e0a7a (127.29.194.2:36405)
I20260807 15:11:59.455433 9437 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9e534f05468e48dca3301b0d578bbf05" candidate_uuid: "86b01224f5b34c19a27d23a31ab7306a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c8b57fd081d41bb8f74cf365d3e0a7a"
I20260807 15:11:59.455530 9437 raft_consensus.cc:3037] T 9e534f05468e48dca3301b0d578bbf05 P 7c8b57fd081d41bb8f74cf365d3e0a7a [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:11:59.455487 9241 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9e534f05468e48dca3301b0d578bbf05" candidate_uuid: "86b01224f5b34c19a27d23a31ab7306a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c708c44654e461099098050b30d16f0"
I20260807 15:11:59.455569 9241 raft_consensus.cc:3037] T 9e534f05468e48dca3301b0d578bbf05 P 1c708c44654e461099098050b30d16f0 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:11:59.456259 9241 raft_consensus.cc:2454] T 9e534f05468e48dca3301b0d578bbf05 P 1c708c44654e461099098050b30d16f0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 86b01224f5b34c19a27d23a31ab7306a in term 1.
I20260807 15:11:59.456319 9437 raft_consensus.cc:2454] T 9e534f05468e48dca3301b0d578bbf05 P 7c8b57fd081d41bb8f74cf365d3e0a7a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 86b01224f5b34c19a27d23a31ab7306a in term 1.
I20260807 15:11:59.456455 9550 leader_election.cc:302] T 9e534f05468e48dca3301b0d578bbf05 P 86b01224f5b34c19a27d23a31ab7306a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1c708c44654e461099098050b30d16f0, 86b01224f5b34c19a27d23a31ab7306a; no voters:
I20260807 15:11:59.456559 9836 raft_consensus.cc:2781] T 9e534f05468e48dca3301b0d578bbf05 P 86b01224f5b34c19a27d23a31ab7306a [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:11:59.456616 9836 raft_consensus.cc:686] T 9e534f05468e48dca3301b0d578bbf05 P 86b01224f5b34c19a27d23a31ab7306a [term 1 LEADER]: Becoming Leader. State: Replica: 86b01224f5b34c19a27d23a31ab7306a, State: Running, Role: LEADER
I20260807 15:11:59.456691 9836 consensus_queue.cc:237] T 9e534f05468e48dca3301b0d578bbf05 P 86b01224f5b34c19a27d23a31ab7306a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c708c44654e461099098050b30d16f0" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 36541 } } peers { permanent_uuid: "7c8b57fd081d41bb8f74cf365d3e0a7a" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 36405 } } peers { permanent_uuid: "86b01224f5b34c19a27d23a31ab7306a" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 44857 } }
I20260807 15:11:59.457263 9008 catalog_manager.cc:5495] T 9e534f05468e48dca3301b0d578bbf05 P 86b01224f5b34c19a27d23a31ab7306a reported cstate change: term changed from 0 to 1, leader changed from <none> to 86b01224f5b34c19a27d23a31ab7306a (127.29.194.3). New cstate: current_term: 1 leader_uuid: "86b01224f5b34c19a27d23a31ab7306a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c708c44654e461099098050b30d16f0" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 36541 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7c8b57fd081d41bb8f74cf365d3e0a7a" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 36405 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "86b01224f5b34c19a27d23a31ab7306a" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 44857 } health_report { overall_health: HEALTHY } } }
I20260807 15:11:59.496066 9814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:59.493698 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46384 (local address 127.29.194.3:44857)
0807 15:11:59.493748 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:59.493750 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:59.493776 (+ 26us) server_negotiation.cc:407] Connection header received
0807 15:11:59.493841 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:59.493842 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:59.493851 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:59.493896 (+ 45us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 15:11:59.494054 (+ 158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:59.494533 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:59.495332 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:59.495583 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:59.495639 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:59.495762 (+ 123us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 15:11:59.495792 (+ 30us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 15:11:59.495837 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0807 15:11:59.495945 (+ 108us) server_negotiation.cc:299] Negotiation successful
0807 15:11:59.495967 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260807 15:11:59.496898 9241 raft_consensus.cc:1261] T 9e534f05468e48dca3301b0d578bbf05 P 1c708c44654e461099098050b30d16f0 [term 1 FOLLOWER]: Refusing update from remote peer 86b01224f5b34c19a27d23a31ab7306a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:11:59.496905 9437 raft_consensus.cc:1261] T 9e534f05468e48dca3301b0d578bbf05 P 7c8b57fd081d41bb8f74cf365d3e0a7a [term 1 FOLLOWER]: Refusing update from remote peer 86b01224f5b34c19a27d23a31ab7306a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:11:59.497160 9836 consensus_queue.cc:1035] T 9e534f05468e48dca3301b0d578bbf05 P 86b01224f5b34c19a27d23a31ab7306a [LEADER]: Connected to new peer: Peer: permanent_uuid: "1c708c44654e461099098050b30d16f0" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 36541 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:11:59.497279 9755 consensus_queue.cc:1035] T 9e534f05468e48dca3301b0d578bbf05 P 86b01224f5b34c19a27d23a31ab7306a [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c8b57fd081d41bb8f74cf365d3e0a7a" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 36405 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:11:59.499688 9009 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56754:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 15:11:59.499778 9009 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56754:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 15:11:59.500357 9009 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f743aff5796d4ca99c67bbf7b9b722d6: Sending DeleteTablet for 3 replicas of tablet 9e534f05468e48dca3301b0d578bbf05
I20260807 15:11:59.500684 9612 tablet_service.cc:1514] Processing DeleteTablet for tablet 9e534f05468e48dca3301b0d578bbf05 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:11:59 UTC) from {username='oryx', principal='oryx/127.29.194.62@KRBTEST.COM'} at 127.0.0.1:46368
I20260807 15:11:59.500731 9410 tablet_service.cc:1514] Processing DeleteTablet for tablet 9e534f05468e48dca3301b0d578bbf05 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:11:59 UTC) from {username='oryx', principal='oryx/127.29.194.62@KRBTEST.COM'} at 127.0.0.1:35692
I20260807 15:11:59.500715 9214 tablet_service.cc:1514] Processing DeleteTablet for tablet 9e534f05468e48dca3301b0d578bbf05 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:11:59 UTC) from {username='oryx', principal='oryx/127.29.194.62@KRBTEST.COM'} at 127.0.0.1:34040
I20260807 15:11:59.500883 9839 tablet_replica.cc:330] T 9e534f05468e48dca3301b0d578bbf05 P 7c8b57fd081d41bb8f74cf365d3e0a7a: stopping tablet replica
I20260807 15:11:59.500883 9838 tablet_replica.cc:330] T 9e534f05468e48dca3301b0d578bbf05 P 1c708c44654e461099098050b30d16f0: stopping tablet replica
I20260807 15:11:59.500902 9843 tablet_replica.cc:330] T 9e534f05468e48dca3301b0d578bbf05 P 86b01224f5b34c19a27d23a31ab7306a: stopping tablet replica
I20260807 15:11:59.500831 9009 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56754:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 15:11:59.500949 9839 raft_consensus.cc:2229] T 9e534f05468e48dca3301b0d578bbf05 P 7c8b57fd081d41bb8f74cf365d3e0a7a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 15:11:59.500968 9838 raft_consensus.cc:2229] T 9e534f05468e48dca3301b0d578bbf05 P 1c708c44654e461099098050b30d16f0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 15:11:59.500975 9843 raft_consensus.cc:2229] T 9e534f05468e48dca3301b0d578bbf05 P 86b01224f5b34c19a27d23a31ab7306a [term 1 LEADER]: Raft consensus shutting down.
W20260807 15:11:59.500984 9009 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 15:11:59.500998 9839 raft_consensus.cc:2258] T 9e534f05468e48dca3301b0d578bbf05 P 7c8b57fd081d41bb8f74cf365d3e0a7a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:11:59.501034 9838 raft_consensus.cc:2258] T 9e534f05468e48dca3301b0d578bbf05 P 1c708c44654e461099098050b30d16f0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:11:59.501075 9843 raft_consensus.cc:2258] T 9e534f05468e48dca3301b0d578bbf05 P 86b01224f5b34c19a27d23a31ab7306a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:11:59.501499 9838 ts_tablet_manager.cc:1905] T 9e534f05468e48dca3301b0d578bbf05 P 1c708c44654e461099098050b30d16f0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:11:59.501746 9843 ts_tablet_manager.cc:1905] T 9e534f05468e48dca3301b0d578bbf05 P 86b01224f5b34c19a27d23a31ab7306a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:11:59.503162 9838 ts_tablet_manager.cc:1918] T 9e534f05468e48dca3301b0d578bbf05 P 1c708c44654e461099098050b30d16f0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 15:11:59.503176 9612 tablet_service.cc:1467] Processing CreateTablet for tablet 001cf21f591c4f1e83a51d8ff9abf2f6 (DEFAULT_TABLE table=test-table [id=ec17ea4acf3f48fb92725a9d78e8e275]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:11:59.503175 9410 tablet_service.cc:1467] Processing CreateTablet for tablet 001cf21f591c4f1e83a51d8ff9abf2f6 (DEFAULT_TABLE table=test-table [id=ec17ea4acf3f48fb92725a9d78e8e275]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:11:59.503199 9843 ts_tablet_manager.cc:1918] T 9e534f05468e48dca3301b0d578bbf05 P 86b01224f5b34c19a27d23a31ab7306a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 15:11:59.503227 9838 log.cc:1196] T 9e534f05468e48dca3301b0d578bbf05 P 1c708c44654e461099098050b30d16f0: Deleting WAL directory at /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115480851507-30472-0/minicluster-data/ts-0/wal/wals/9e534f05468e48dca3301b0d578bbf05
I20260807 15:11:59.503245 9843 log.cc:1196] T 9e534f05468e48dca3301b0d578bbf05 P 86b01224f5b34c19a27d23a31ab7306a: Deleting WAL directory at /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115480851507-30472-0/minicluster-data/ts-2/wal/wals/9e534f05468e48dca3301b0d578bbf05
I20260807 15:11:59.503281 9612 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 001cf21f591c4f1e83a51d8ff9abf2f6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:11:59.503316 9410 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 001cf21f591c4f1e83a51d8ff9abf2f6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:11:59.503458 9843 ts_tablet_manager.cc:1939] T 9e534f05468e48dca3301b0d578bbf05 P 86b01224f5b34c19a27d23a31ab7306a: Deleting consensus metadata
I20260807 15:11:59.503475 9838 ts_tablet_manager.cc:1939] T 9e534f05468e48dca3301b0d578bbf05 P 1c708c44654e461099098050b30d16f0: Deleting consensus metadata
I20260807 15:11:59.503727 9839 ts_tablet_manager.cc:1905] T 9e534f05468e48dca3301b0d578bbf05 P 7c8b57fd081d41bb8f74cf365d3e0a7a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:11:59.504490 9749 tablet_bootstrap.cc:492] T 001cf21f591c4f1e83a51d8ff9abf2f6 P 86b01224f5b34c19a27d23a31ab7306a: Bootstrap starting.
I20260807 15:11:59.504568 9759 tablet_bootstrap.cc:492] T 001cf21f591c4f1e83a51d8ff9abf2f6 P 7c8b57fd081d41bb8f74cf365d3e0a7a: Bootstrap starting.
I20260807 15:11:59.504798 9749 tablet_bootstrap.cc:654] T 001cf21f591c4f1e83a51d8ff9abf2f6 P 86b01224f5b34c19a27d23a31ab7306a: Neither blocks nor log segments found. Creating new log.
I20260807 15:11:59.504787 8985 catalog_manager.cc:4841] TS 1c708c44654e461099098050b30d16f0 (127.29.194.1:36541): tablet 9e534f05468e48dca3301b0d578bbf05 (table test-table [id=efc700fe31b74a6cbf6217fc8d242c4c]) successfully deleted
I20260807 15:11:59.505139 9839 ts_tablet_manager.cc:1918] T 9e534f05468e48dca3301b0d578bbf05 P 7c8b57fd081d41bb8f74cf365d3e0a7a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 15:11:59.505201 9839 log.cc:1196] T 9e534f05468e48dca3301b0d578bbf05 P 7c8b57fd081d41bb8f74cf365d3e0a7a: Deleting WAL directory at /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115480851507-30472-0/minicluster-data/ts-1/wal/wals/9e534f05468e48dca3301b0d578bbf05
I20260807 15:11:59.505275 9759 tablet_bootstrap.cc:654] T 001cf21f591c4f1e83a51d8ff9abf2f6 P 7c8b57fd081d41bb8f74cf365d3e0a7a: Neither blocks nor log segments found. Creating new log.
I20260807 15:11:59.505403 9839 ts_tablet_manager.cc:1939] T 9e534f05468e48dca3301b0d578bbf05 P 7c8b57fd081d41bb8f74cf365d3e0a7a: Deleting consensus metadata
I20260807 15:11:59.505668 9214 tablet_service.cc:1467] Processing CreateTablet for tablet 001cf21f591c4f1e83a51d8ff9abf2f6 (DEFAULT_TABLE table=test-table [id=ec17ea4acf3f48fb92725a9d78e8e275]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:11:59.505772 9214 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 001cf21f591c4f1e83a51d8ff9abf2f6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:11:59.506007 9759 tablet_bootstrap.cc:492] T 001cf21f591c4f1e83a51d8ff9abf2f6 P 7c8b57fd081d41bb8f74cf365d3e0a7a: No bootstrap required, opened a new log
I20260807 15:11:59.506062 9759 ts_tablet_manager.cc:1397] T 001cf21f591c4f1e83a51d8ff9abf2f6 P 7c8b57fd081d41bb8f74cf365d3e0a7a: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260807 15:11:59.506208 9759 raft_consensus.cc:348] T 001cf21f591c4f1e83a51d8ff9abf2f6 P 7c8b57fd081d41bb8f74cf365d3e0a7a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c708c44654e461099098050b30d16f0" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 36541 } } peers { permanent_uuid: "7c8b57fd081d41bb8f74cf365d3e0a7a" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 36405 } } peers { permanent_uuid: "86b01224f5b34c19a27d23a31ab7306a" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 44857 } }
I20260807 15:11:59.506279 9759 raft_consensus.cc:374] T 001cf21f591c4f1e83a51d8ff9abf2f6 P 7c8b57fd081d41bb8f74cf365d3e0a7a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:11:59.506297 9759 raft_consensus.cc:729] T 001cf21f591c4f1e83a51d8ff9abf2f6 P 7c8b57fd081d41bb8f74cf365d3e0a7a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c8b57fd081d41bb8f74cf365d3e0a7a, State: Initialized, Role: FOLLOWER
I20260807 15:11:59.506343 9759 consensus_queue.cc:260] T 001cf21f591c4f1e83a51d8ff9abf2f6 P 7c8b57fd081d41bb8f74cf365d3e0a7a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c708c44654e461099098050b30d16f0" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 36541 } } peers { permanent_uuid: "7c8b57fd081d41bb8f74cf365d3e0a7a" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 36405 } } peers { permanent_uuid: "86b01224f5b34c19a27d23a31ab7306a" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 44857 } }
I20260807 15:11:59.506434 9759 ts_tablet_manager.cc:1428] T 001cf21f591c4f1e83a51d8ff9abf2f6 P 7c8b57fd081d41bb8f74cf365d3e0a7a: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260807 15:11:59.506795 9750 tablet_bootstrap.cc:492] T 001cf21f591c4f1e83a51d8ff9abf2f6 P 1c708c44654e461099098050b30d16f0: Bootstrap starting.
I20260807 15:11:59.507112 9750 tablet_bootstrap.cc:654] T 001cf21f591c4f1e83a51d8ff9abf2f6 P 1c708c44654e461099098050b30d16f0: Neither blocks nor log segments found. Creating new log.
I20260807 15:11:59.507115 8986 catalog_manager.cc:4841] TS 7c8b57fd081d41bb8f74cf365d3e0a7a (127.29.194.2:36405): tablet 9e534f05468e48dca3301b0d578bbf05 (table test-table [id=efc700fe31b74a6cbf6217fc8d242c4c]) successfully deleted
I20260807 15:11:59.507426 8985 catalog_manager.cc:4841] TS 86b01224f5b34c19a27d23a31ab7306a (127.29.194.3:44857): tablet 9e534f05468e48dca3301b0d578bbf05 (table test-table [id=efc700fe31b74a6cbf6217fc8d242c4c]) successfully deleted
I20260807 15:11:59.507629 9750 tablet_bootstrap.cc:492] T 001cf21f591c4f1e83a51d8ff9abf2f6 P 1c708c44654e461099098050b30d16f0: No bootstrap required, opened a new log
I20260807 15:11:59.507673 9750 ts_tablet_manager.cc:1397] T 001cf21f591c4f1e83a51d8ff9abf2f6 P 1c708c44654e461099098050b30d16f0: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260807 15:11:59.507771 9749 tablet_bootstrap.cc:492] T 001cf21f591c4f1e83a51d8ff9abf2f6 P 86b01224f5b34c19a27d23a31ab7306a: No bootstrap required, opened a new log
I20260807 15:11:59.507777 9750 raft_consensus.cc:348] T 001cf21f591c4f1e83a51d8ff9abf2f6 P 1c708c44654e461099098050b30d16f0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c708c44654e461099098050b30d16f0" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 36541 } } peers { permanent_uuid: "7c8b57fd081d41bb8f74cf365d3e0a7a" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 36405 } } peers { permanent_uuid: "86b01224f5b34c19a27d23a31ab7306a" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 44857 } }
I20260807 15:11:59.507817 9749 ts_tablet_manager.cc:1397] T 001cf21f591c4f1e83a51d8ff9abf2f6 P 86b01224f5b34c19a27d23a31ab7306a: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20260807 15:11:59.507833 9750 raft_consensus.cc:374] T 001cf21f591c4f1e83a51d8ff9abf2f6 P 1c708c44654e461099098050b30d16f0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:11:59.507858 9750 raft_consensus.cc:729] T 001cf21f591c4f1e83a51d8ff9abf2f6 P 1c708c44654e461099098050b30d16f0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1c708c44654e461099098050b30d16f0, State: Initialized, Role: FOLLOWER
I20260807 15:11:59.507905 9750 consensus_queue.cc:260] T 001cf21f591c4f1e83a51d8ff9abf2f6 P 1c708c44654e461099098050b30d16f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c708c44654e461099098050b30d16f0" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 36541 } } peers { permanent_uuid: "7c8b57fd081d41bb8f74cf365d3e0a7a" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 36405 } } peers { permanent_uuid: "86b01224f5b34c19a27d23a31ab7306a" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 44857 } }
I20260807 15:11:59.507927 9749 raft_consensus.cc:348] T 001cf21f591c4f1e83a51d8ff9abf2f6 P 86b01224f5b34c19a27d23a31ab7306a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c708c44654e461099098050b30d16f0" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 36541 } } peers { permanent_uuid: "7c8b57fd081d41bb8f74cf365d3e0a7a" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 36405 } } peers { permanent_uuid: "86b01224f5b34c19a27d23a31ab7306a" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 44857 } }
I20260807 15:11:59.507997 9750 ts_tablet_manager.cc:1428] T 001cf21f591c4f1e83a51d8ff9abf2f6 P 1c708c44654e461099098050b30d16f0: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260807 15:11:59.508006 9749 raft_consensus.cc:374] T 001cf21f591c4f1e83a51d8ff9abf2f6 P 86b01224f5b34c19a27d23a31ab7306a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:11:59.508028 9749 raft_consensus.cc:729] T 001cf21f591c4f1e83a51d8ff9abf2f6 P 86b01224f5b34c19a27d23a31ab7306a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 86b01224f5b34c19a27d23a31ab7306a, State: Initialized, Role: FOLLOWER
I20260807 15:11:59.508072 9749 consensus_queue.cc:260] T 001cf21f591c4f1e83a51d8ff9abf2f6 P 86b01224f5b34c19a27d23a31ab7306a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c708c44654e461099098050b30d16f0" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 36541 } } peers { permanent_uuid: "7c8b57fd081d41bb8f74cf365d3e0a7a" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 36405 } } peers { permanent_uuid: "86b01224f5b34c19a27d23a31ab7306a" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 44857 } }
I20260807 15:11:59.508157 9749 ts_tablet_manager.cc:1428] T 001cf21f591c4f1e83a51d8ff9abf2f6 P 86b01224f5b34c19a27d23a31ab7306a: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260807 15:11:59.819393 9834 raft_consensus.cc:482] T 001cf21f591c4f1e83a51d8ff9abf2f6 P 1c708c44654e461099098050b30d16f0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 15:11:59.819649 9834 raft_consensus.cc:504] T 001cf21f591c4f1e83a51d8ff9abf2f6 P 1c708c44654e461099098050b30d16f0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c708c44654e461099098050b30d16f0" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 36541 } } peers { permanent_uuid: "7c8b57fd081d41bb8f74cf365d3e0a7a" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 36405 } } peers { permanent_uuid: "86b01224f5b34c19a27d23a31ab7306a" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 44857 } }
I20260807 15:11:59.819893 9834 leader_election.cc:288] T 001cf21f591c4f1e83a51d8ff9abf2f6 P 1c708c44654e461099098050b30d16f0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7c8b57fd081d41bb8f74cf365d3e0a7a (127.29.194.2:36405), 86b01224f5b34c19a27d23a31ab7306a (127.29.194.3:44857)
I20260807 15:11:59.820344 9636 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "001cf21f591c4f1e83a51d8ff9abf2f6" candidate_uuid: "1c708c44654e461099098050b30d16f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "86b01224f5b34c19a27d23a31ab7306a" is_pre_election: true
I20260807 15:11:59.820344 9437 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "001cf21f591c4f1e83a51d8ff9abf2f6" candidate_uuid: "1c708c44654e461099098050b30d16f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c8b57fd081d41bb8f74cf365d3e0a7a" is_pre_election: true
I20260807 15:11:59.820511 9636 raft_consensus.cc:2454] T 001cf21f591c4f1e83a51d8ff9abf2f6 P 86b01224f5b34c19a27d23a31ab7306a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1c708c44654e461099098050b30d16f0 in term 0.
I20260807 15:11:59.820523 9437 raft_consensus.cc:2454] T 001cf21f591c4f1e83a51d8ff9abf2f6 P 7c8b57fd081d41bb8f74cf365d3e0a7a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1c708c44654e461099098050b30d16f0 in term 0.
I20260807 15:11:59.820717 9149 leader_election.cc:302] T 001cf21f591c4f1e83a51d8ff9abf2f6 P 1c708c44654e461099098050b30d16f0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 1c708c44654e461099098050b30d16f0, 7c8b57fd081d41bb8f74cf365d3e0a7a, 86b01224f5b34c19a27d23a31ab7306a; no voters:
I20260807 15:11:59.820897 9834 raft_consensus.cc:2781] T 001cf21f591c4f1e83a51d8ff9abf2f6 P 1c708c44654e461099098050b30d16f0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 15:11:59.820957 9834 raft_consensus.cc:482] T 001cf21f591c4f1e83a51d8ff9abf2f6 P 1c708c44654e461099098050b30d16f0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 15:11:59.820986 9834 raft_consensus.cc:3037] T 001cf21f591c4f1e83a51d8ff9abf2f6 P 1c708c44654e461099098050b30d16f0 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:11:59.822008 9834 raft_consensus.cc:504] T 001cf21f591c4f1e83a51d8ff9abf2f6 P 1c708c44654e461099098050b30d16f0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c708c44654e461099098050b30d16f0" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 36541 } } peers { permanent_uuid: "7c8b57fd081d41bb8f74cf365d3e0a7a" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 36405 } } peers { permanent_uuid: "86b01224f5b34c19a27d23a31ab7306a" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 44857 } }
I20260807 15:11:59.822190 9834 leader_election.cc:288] T 001cf21f591c4f1e83a51d8ff9abf2f6 P 1c708c44654e461099098050b30d16f0 [CANDIDATE]: Term 1 election: Requested vote from peers 7c8b57fd081d41bb8f74cf365d3e0a7a (127.29.194.2:36405), 86b01224f5b34c19a27d23a31ab7306a (127.29.194.3:44857)
I20260807 15:11:59.822454 9437 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "001cf21f591c4f1e83a51d8ff9abf2f6" candidate_uuid: "1c708c44654e461099098050b30d16f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c8b57fd081d41bb8f74cf365d3e0a7a"
I20260807 15:11:59.822482 9636 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "001cf21f591c4f1e83a51d8ff9abf2f6" candidate_uuid: "1c708c44654e461099098050b30d16f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "86b01224f5b34c19a27d23a31ab7306a"
I20260807 15:11:59.822551 9437 raft_consensus.cc:3037] T 001cf21f591c4f1e83a51d8ff9abf2f6 P 7c8b57fd081d41bb8f74cf365d3e0a7a [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:11:59.822573 9636 raft_consensus.cc:3037] T 001cf21f591c4f1e83a51d8ff9abf2f6 P 86b01224f5b34c19a27d23a31ab7306a [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:11:59.823316 9437 raft_consensus.cc:2454] T 001cf21f591c4f1e83a51d8ff9abf2f6 P 7c8b57fd081d41bb8f74cf365d3e0a7a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1c708c44654e461099098050b30d16f0 in term 1.
I20260807 15:11:59.823336 9636 raft_consensus.cc:2454] T 001cf21f591c4f1e83a51d8ff9abf2f6 P 86b01224f5b34c19a27d23a31ab7306a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1c708c44654e461099098050b30d16f0 in term 1.
I20260807 15:11:59.823596 9151 leader_election.cc:302] T 001cf21f591c4f1e83a51d8ff9abf2f6 P 1c708c44654e461099098050b30d16f0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1c708c44654e461099098050b30d16f0, 7c8b57fd081d41bb8f74cf365d3e0a7a; no voters:
I20260807 15:11:59.823809 9834 raft_consensus.cc:2781] T 001cf21f591c4f1e83a51d8ff9abf2f6 P 1c708c44654e461099098050b30d16f0 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:11:59.823953 9834 raft_consensus.cc:686] T 001cf21f591c4f1e83a51d8ff9abf2f6 P 1c708c44654e461099098050b30d16f0 [term 1 LEADER]: Becoming Leader. State: Replica: 1c708c44654e461099098050b30d16f0, State: Running, Role: LEADER
I20260807 15:11:59.824012 9834 consensus_queue.cc:237] T 001cf21f591c4f1e83a51d8ff9abf2f6 P 1c708c44654e461099098050b30d16f0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c708c44654e461099098050b30d16f0" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 36541 } } peers { permanent_uuid: "7c8b57fd081d41bb8f74cf365d3e0a7a" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 36405 } } peers { permanent_uuid: "86b01224f5b34c19a27d23a31ab7306a" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 44857 } }
I20260807 15:11:59.824700 9008 catalog_manager.cc:5495] T 001cf21f591c4f1e83a51d8ff9abf2f6 P 1c708c44654e461099098050b30d16f0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1c708c44654e461099098050b30d16f0 (127.29.194.1). New cstate: current_term: 1 leader_uuid: "1c708c44654e461099098050b30d16f0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c708c44654e461099098050b30d16f0" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 36541 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7c8b57fd081d41bb8f74cf365d3e0a7a" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 36405 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "86b01224f5b34c19a27d23a31ab7306a" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 44857 } health_report { overall_health: UNKNOWN } } }
I20260807 15:11:59.869916 9883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:59.867669 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55626 (local address 127.29.194.1:36541)
0807 15:11:59.867796 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:59.867798 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:59.867805 (+ 7us) server_negotiation.cc:407] Connection header received
0807 15:11:59.867827 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:59.867827 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:59.867837 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:59.867892 (+ 55us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 15:11:59.868038 (+ 146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:59.868539 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:59.869241 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:59.869479 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:59.869536 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:59.869669 (+ 133us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 15:11:59.869705 (+ 36us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 15:11:59.869748 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0807 15:11:59.869803 (+ 55us) server_negotiation.cc:299] Negotiation successful
0807 15:11:59.869826 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":109,"thread_start_us":66,"threads_started":1}
I20260807 15:11:59.875422 9891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:59.872500 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.3:44857 (local address 127.29.194.1:47755)
0807 15:11:59.872647 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0807 15:11:59.872658 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 15:11:59.872729 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:11:59.872875 (+ 146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:11:59.872876 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:11:59.872878 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 15:11:59.872931 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:59.872933 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:59.873704 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:59.873706 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:59.874430 (+ 724us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:59.874434 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:59.874894 (+ 460us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:59.874896 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:59.874965 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:59.874983 (+ 18us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 15:11:59.875253 (+ 270us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 15:11:59.875254 (+ 1us) client_negotiation.cc:769] Sending connection context
0807 15:11:59.875291 (+ 37us) client_negotiation.cc:240] Negotiation successful
0807 15:11:59.875345 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":68,"threads_started":1}
I20260807 15:11:59.875424 9814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:59.872607 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.194.1:47755 (local address 127.29.194.3:44857)
0807 15:11:59.872649 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:59.872651 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:59.872701 (+ 50us) server_negotiation.cc:407] Connection header received
0807 15:11:59.872792 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:59.872793 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:59.872807 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:59.872865 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 15:11:59.872972 (+ 107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:59.873525 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:59.874576 (+ 1051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:59.874775 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:59.874848 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:59.875137 (+ 289us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 15:11:59.875181 (+ 44us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 15:11:59.875244 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0807 15:11:59.875298 (+ 54us) server_negotiation.cc:299] Negotiation successful
0807 15:11:59.875340 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260807 15:11:59.877807 9437 raft_consensus.cc:1261] T 001cf21f591c4f1e83a51d8ff9abf2f6 P 7c8b57fd081d41bb8f74cf365d3e0a7a [term 1 FOLLOWER]: Refusing update from remote peer 1c708c44654e461099098050b30d16f0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:11:59.877856 9636 raft_consensus.cc:1261] T 001cf21f591c4f1e83a51d8ff9abf2f6 P 86b01224f5b34c19a27d23a31ab7306a [term 1 FOLLOWER]: Refusing update from remote peer 1c708c44654e461099098050b30d16f0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:11:59.878047 9834 consensus_queue.cc:1035] T 001cf21f591c4f1e83a51d8ff9abf2f6 P 1c708c44654e461099098050b30d16f0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c8b57fd081d41bb8f74cf365d3e0a7a" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 36405 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:11:59.878182 9879 consensus_queue.cc:1035] T 001cf21f591c4f1e83a51d8ff9abf2f6 P 1c708c44654e461099098050b30d16f0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "86b01224f5b34c19a27d23a31ab7306a" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 44857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:11:59.885776 9883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:59.882885 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.194.3:58257 (local address 127.29.194.1:36541)
0807 15:11:59.882929 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:59.882931 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:59.883115 (+ 184us) server_negotiation.cc:407] Connection header received
0807 15:11:59.883243 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:59.883244 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:59.883257 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:59.883305 (+ 48us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 15:11:59.883443 (+ 138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:59.884232 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:59.885172 (+ 940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:59.885360 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:59.885415 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:59.885581 (+ 166us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 15:11:59.885610 (+ 29us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 15:11:59.885648 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0807 15:11:59.885686 (+ 38us) server_negotiation.cc:299] Negotiation successful
0807 15:11:59.885706 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260807 15:11:59.885803 9898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:59.882924 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.1:36541 (local address 127.29.194.3:58257)
0807 15:11:59.883038 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0807 15:11:59.883048 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 15:11:59.883145 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:11:59.883310 (+ 165us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:11:59.883312 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:11:59.883314 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 15:11:59.883369 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:59.883370 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:59.884324 (+ 954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:59.884325 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:59.885098 (+ 773us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:59.885101 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:59.885429 (+ 328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:59.885430 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:59.885512 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:59.885518 (+ 6us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 15:11:59.885654 (+ 136us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 15:11:59.885655 (+ 1us) client_negotiation.cc:769] Sending connection context
0807 15:11:59.885683 (+ 28us) client_negotiation.cc:240] Negotiation successful
0807 15:11:59.885720 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":73,"threads_started":1}
I20260807 15:11:59.892795 9009 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56754:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 15:11:59.892908 9009 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56754:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 15:11:59.893841 9009 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f743aff5796d4ca99c67bbf7b9b722d6: Sending DeleteTablet for 3 replicas of tablet 001cf21f591c4f1e83a51d8ff9abf2f6
I20260807 15:11:59.894364 9410 tablet_service.cc:1514] Processing DeleteTablet for tablet 001cf21f591c4f1e83a51d8ff9abf2f6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:11:59 UTC) from {username='oryx', principal='oryx/127.29.194.62@KRBTEST.COM'} at 127.0.0.1:35692
I20260807 15:11:59.894397 9612 tablet_service.cc:1514] Processing DeleteTablet for tablet 001cf21f591c4f1e83a51d8ff9abf2f6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:11:59 UTC) from {username='oryx', principal='oryx/127.29.194.62@KRBTEST.COM'} at 127.0.0.1:46368
I20260807 15:11:59.894526 9214 tablet_service.cc:1514] Processing DeleteTablet for tablet 001cf21f591c4f1e83a51d8ff9abf2f6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:11:59 UTC) from {username='oryx', principal='oryx/127.29.194.62@KRBTEST.COM'} at 127.0.0.1:34040
I20260807 15:11:59.894531 9843 tablet_replica.cc:330] T 001cf21f591c4f1e83a51d8ff9abf2f6 P 86b01224f5b34c19a27d23a31ab7306a: stopping tablet replica
I20260807 15:11:59.894619 9838 tablet_replica.cc:330] T 001cf21f591c4f1e83a51d8ff9abf2f6 P 1c708c44654e461099098050b30d16f0: stopping tablet replica
I20260807 15:11:59.894661 9843 raft_consensus.cc:2229] T 001cf21f591c4f1e83a51d8ff9abf2f6 P 86b01224f5b34c19a27d23a31ab7306a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 15:11:59.894694 9838 raft_consensus.cc:2229] T 001cf21f591c4f1e83a51d8ff9abf2f6 P 1c708c44654e461099098050b30d16f0 [term 1 LEADER]: Raft consensus shutting down.
I20260807 15:11:59.894724 9843 raft_consensus.cc:2258] T 001cf21f591c4f1e83a51d8ff9abf2f6 P 86b01224f5b34c19a27d23a31ab7306a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:11:59.894766 9838 raft_consensus.cc:2258] T 001cf21f591c4f1e83a51d8ff9abf2f6 P 1c708c44654e461099098050b30d16f0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:11:59.894824 9839 tablet_replica.cc:330] T 001cf21f591c4f1e83a51d8ff9abf2f6 P 7c8b57fd081d41bb8f74cf365d3e0a7a: stopping tablet replica
I20260807 15:11:59.894909 9839 raft_consensus.cc:2229] T 001cf21f591c4f1e83a51d8ff9abf2f6 P 7c8b57fd081d41bb8f74cf365d3e0a7a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 15:11:59.894955 9839 raft_consensus.cc:2258] T 001cf21f591c4f1e83a51d8ff9abf2f6 P 7c8b57fd081d41bb8f74cf365d3e0a7a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:11:59.895140 9838 ts_tablet_manager.cc:1905] T 001cf21f591c4f1e83a51d8ff9abf2f6 P 1c708c44654e461099098050b30d16f0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:11:59.895211 9843 ts_tablet_manager.cc:1905] T 001cf21f591c4f1e83a51d8ff9abf2f6 P 86b01224f5b34c19a27d23a31ab7306a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:11:59.895339 30472 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7fZSIL/build/release/bin/kudu with pid 9122
I20260807 15:11:59.898784 9843 ts_tablet_manager.cc:1918] T 001cf21f591c4f1e83a51d8ff9abf2f6 P 86b01224f5b34c19a27d23a31ab7306a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 15:11:59.898869 9843 log.cc:1196] T 001cf21f591c4f1e83a51d8ff9abf2f6 P 86b01224f5b34c19a27d23a31ab7306a: Deleting WAL directory at /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115480851507-30472-0/minicluster-data/ts-2/wal/wals/001cf21f591c4f1e83a51d8ff9abf2f6
I20260807 15:11:59.899080 9843 ts_tablet_manager.cc:1939] T 001cf21f591c4f1e83a51d8ff9abf2f6 P 86b01224f5b34c19a27d23a31ab7306a: Deleting consensus metadata
I20260807 15:11:59.899478 8985 catalog_manager.cc:4841] TS 86b01224f5b34c19a27d23a31ab7306a (127.29.194.3:44857): tablet 001cf21f591c4f1e83a51d8ff9abf2f6 (table test-table [id=ec17ea4acf3f48fb92725a9d78e8e275]) successfully deleted
I20260807 15:11:59.901926 9839 ts_tablet_manager.cc:1905] T 001cf21f591c4f1e83a51d8ff9abf2f6 P 7c8b57fd081d41bb8f74cf365d3e0a7a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:11:59.903534 9839 ts_tablet_manager.cc:1918] T 001cf21f591c4f1e83a51d8ff9abf2f6 P 7c8b57fd081d41bb8f74cf365d3e0a7a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 15:11:59.903599 9839 log.cc:1196] T 001cf21f591c4f1e83a51d8ff9abf2f6 P 7c8b57fd081d41bb8f74cf365d3e0a7a: Deleting WAL directory at /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115480851507-30472-0/minicluster-data/ts-1/wal/wals/001cf21f591c4f1e83a51d8ff9abf2f6
I20260807 15:11:59.903795 9839 ts_tablet_manager.cc:1939] T 001cf21f591c4f1e83a51d8ff9abf2f6 P 7c8b57fd081d41bb8f74cf365d3e0a7a: Deleting consensus metadata
I20260807 15:11:59.905414 8986 catalog_manager.cc:4841] TS 7c8b57fd081d41bb8f74cf365d3e0a7a (127.29.194.2:36405): tablet 001cf21f591c4f1e83a51d8ff9abf2f6 (table test-table [id=ec17ea4acf3f48fb92725a9d78e8e275]) successfully deleted
W20260807 15:11:59.912348 8985 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.29.194.1:36541 (error 108)
I20260807 15:11:59.913149 30472 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7fZSIL/build/release/bin/kudu with pid 9318
I20260807 15:11:59.913187 9901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:59.912899 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.1:36541 (local address 127.0.0.1:55628)
0807 15:11:59.913056 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0807 15:11:59.913130 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.194.1:36541: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":55,"threads_started":1}
W20260807 15:11:59.919870 8985 catalog_manager.cc:4584] TS 1c708c44654e461099098050b30d16f0 (127.29.194.1:36541): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 001cf21f591c4f1e83a51d8ff9abf2f6: Network error: Client connection negotiation failed: client connection to 127.29.194.1:36541: connect: Connection refused (error 111)
I20260807 15:11:59.924220 30472 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7fZSIL/build/release/bin/kudu with pid 9522
I20260807 15:11:59.938588 30472 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7fZSIL/build/release/bin/kudu with pid 8956
2026-08-07T15:11:59Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (3623 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786115480851507-30472-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 15:12:00 dist-test-slave-42z9 krb5kdc[9904](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 15:12:00 dist-test-slave-42z9 krb5kdc[9904](info): set up 2 sockets
Aug 07 15:12:00 dist-test-slave-42z9 krb5kdc[9904](info): commencing operation
krb5kdc: starting...
W20260807 15:12:01.110145 30472 mini_kdc.cc:121] Time spent starting KDC: real 1.143s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786115480851507-30472-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786115480851507-30472-0/krb5kdc/test-user.keytab.
Aug 07 15:12:01 dist-test-slave-42z9 krb5kdc[9904](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115521, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T15:12:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T15:12:01Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.194.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.194.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.194.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.62.keytab.
Entry for principal kudu/127.29.194.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.62.keytab.
I20260807 15:12:01.246657 30472 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7fZSIL/build/release/bin/kudu
/tmp/dist-test-task7fZSIL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786115480851507-30472-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786115480851507-30472-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786115480851507-30472-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786115480851507-30472-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.194.62:41791
--webserver_interface=127.29.194.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.62.keytab
--principal=kudu/127.29.194.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.194.20:40011
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.194.62:41791
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786115480851507-30472-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786115480851507-30472-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786115480851507-30472-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:12:01.333851 9922 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:12:01.334091 9922 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:12:01.334124 9922 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:12:01.335399 9922 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:12:01.335461 9922 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:12:01.335480 9922 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:12:01.335497 9922 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:12:01.335513 9922 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:12:01.337646 9922 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.194.20:40011
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786115480851507-30472-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786115480851507-30472-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.194.62:41791
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.62.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.194.62:41791
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786115480851507-30472-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.194.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9922
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786115480851507-30472-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:12:01.338088 9922 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:12:01.338425 9922 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:12:01.344504 9933 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:12:01.344480 9931 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:12:01.344583 9922 server_base.cc:1034] running on GCE node
W20260807 15:12:01.344482 9930 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:12:01.345221 9922 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:12:01.345585 9922 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:12:01.346786 9922 hybrid_clock.cc:648] HybridClock initialized: now 1786115521346763 us; error 40 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-07T15:12:01Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1409 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115480851507-30472-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 15:12:01 dist-test-slave-42z9 krb5kdc[9940](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 15:12:01 dist-test-slave-42z9 krb5kdc[9940](info): set up 2 sockets
Aug 07 15:12:01 dist-test-slave-42z9 krb5kdc[9940](info): commencing operation
krb5kdc: starting...
W20260807 15:12:02.514416 30472 mini_kdc.cc:121] Time spent starting KDC: real 1.138s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115480851507-30472-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115480851507-30472-0/krb5kdc/test-user.keytab.
Aug 07 15:12:02 dist-test-slave-42z9 krb5kdc[9940](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115522, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T15:12:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T15:12:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.194.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.194.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.194.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.62.keytab.
Entry for principal kudu/127.29.194.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.62.keytab.
I20260807 15:12:02.653903 30472 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7fZSIL/build/release/bin/kudu
/tmp/dist-test-task7fZSIL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115480851507-30472-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115480851507-30472-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115480851507-30472-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115480851507-30472-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.194.62:34011
--webserver_interface=127.29.194.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.62.keytab
--principal=kudu/127.29.194.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.194.20:44991
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.194.62:34011
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115480851507-30472-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115480851507-30472-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115480851507-30472-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:12:02.744211 9959 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:12:02.744469 9959 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:12:02.744504 9959 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:12:02.745797 9959 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:12:02.745852 9959 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:12:02.745870 9959 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:12:02.745887 9959 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:12:02.745903 9959 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:12:02.747781 9959 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.194.20:44991
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115480851507-30472-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115480851507-30472-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.194.62:34011
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.62.keytab
--principal=kudu/127.29.194.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.194.62:34011
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115480851507-30472-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.194.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9959
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115480851507-30472-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:12:02.748108 9959 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:12:02.748350 9959 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:12:02.753796 9968 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:12:02.753796 9965 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:12:02.753775 9966 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:12:02.754004 9959 server_base.cc:1034] running on GCE node
I20260807 15:12:02.754338 9959 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:12:02.754658 9959 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:12:02.755897 9959 hybrid_clock.cc:648] HybridClock initialized: now 1786115522755868 us; error 36 us; skew 500 ppm
Aug 07 15:12:02 dist-test-slave-42z9 krb5kdc[9940](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115522, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.194.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:12:02.759128 9959 init.cc:377] Logged in from keytab as kudu/127.29.194.62@KRBTEST.COM (short username kudu)
I20260807 15:12:02.759665 9959 webserver.cc:460] Webserver started at http://127.29.194.62:39211/ using document root <none> and password file <none>
I20260807 15:12:02.759889 9959 fs_manager.cc:359] Metadata directory not provided
I20260807 15:12:02.759943 9959 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:12:02.760083 9959 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:12:02.761292 9959 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115480851507-30472-0/minicluster-data/master-0/data/instance:
uuid: "f0f928f3cb6f4a1490e025078b11f5c3"
format_stamp: "Formatted at 2026-08-07 15:12:02 on dist-test-slave-42z9"
server_key: "9dee93f8b06be22d17313992eeea50e7"
server_key_iv: "b4a48b6fff5f413dcfdcfb739c5b318f"
server_key_version: "encryptionkey@0"
I20260807 15:12:02.761705 9959 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115480851507-30472-0/minicluster-data/master-0/wal/instance:
uuid: "f0f928f3cb6f4a1490e025078b11f5c3"
format_stamp: "Formatted at 2026-08-07 15:12:02 on dist-test-slave-42z9"
server_key: "9dee93f8b06be22d17313992eeea50e7"
server_key_iv: "b4a48b6fff5f413dcfdcfb739c5b318f"
server_key_version: "encryptionkey@0"
I20260807 15:12:02.763286 9959 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:12:02.764366 9980 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:12:02.764638 9959 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:12:02.764750 9959 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115480851507-30472-0/minicluster-data/master-0/data,/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115480851507-30472-0/minicluster-data/master-0/wal
uuid: "f0f928f3cb6f4a1490e025078b11f5c3"
format_stamp: "Formatted at 2026-08-07 15:12:02 on dist-test-slave-42z9"
server_key: "9dee93f8b06be22d17313992eeea50e7"
server_key_iv: "b4a48b6fff5f413dcfdcfb739c5b318f"
server_key_version: "encryptionkey@0"
I20260807 15:12:02.764822 9959 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115480851507-30472-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115480851507-30472-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115480851507-30472-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:12:02.779667 9959 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:12:02.782467 9959 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:12:02.782631 9959 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:12:02.786919 9959 rpc_server.cc:307] RPC server started. Bound to: 127.29.194.62:34011
I20260807 15:12:02.786954 10061 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.194.62:34011 every 8 connection(s)
I20260807 15:12:02.787298 9959 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115480851507-30472-0/minicluster-data/master-0/data/info.pb
I20260807 15:12:02.788642 10062 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:12:02.791507 10062 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f0f928f3cb6f4a1490e025078b11f5c3: Bootstrap starting.
I20260807 15:12:02.791612 30472 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7fZSIL/build/release/bin/kudu as pid 9959
I20260807 15:12:02.791726 30472 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115480851507-30472-0/minicluster-data/master-0/wal/instance
I20260807 15:12:02.791908 30472 external_mini_cluster.cc:1428] Setting key b7c4b9d29a41c8073d1b13b8c4c07acd
I20260807 15:12:02.792227 10062 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f0f928f3cb6f4a1490e025078b11f5c3: Neither blocks nor log segments found. Creating new log.
I20260807 15:12:02.792568 10062 log.cc:824] T 00000000000000000000000000000000 P f0f928f3cb6f4a1490e025078b11f5c3: Log is configured to *not* fsync() on all Append() calls
I20260807 15:12:02.793292 10062 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f0f928f3cb6f4a1490e025078b11f5c3: No bootstrap required, opened a new log
I20260807 15:12:02.795574 10062 raft_consensus.cc:348] T 00000000000000000000000000000000 P f0f928f3cb6f4a1490e025078b11f5c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f0f928f3cb6f4a1490e025078b11f5c3" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 34011 } }
I20260807 15:12:02.795770 10062 raft_consensus.cc:374] T 00000000000000000000000000000000 P f0f928f3cb6f4a1490e025078b11f5c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:12:02.795806 10062 raft_consensus.cc:729] T 00000000000000000000000000000000 P f0f928f3cb6f4a1490e025078b11f5c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f0f928f3cb6f4a1490e025078b11f5c3, State: Initialized, Role: FOLLOWER
I20260807 15:12:02.795934 10062 consensus_queue.cc:260] T 00000000000000000000000000000000 P f0f928f3cb6f4a1490e025078b11f5c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f0f928f3cb6f4a1490e025078b11f5c3" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 34011 } }
I20260807 15:12:02.796016 10062 raft_consensus.cc:388] T 00000000000000000000000000000000 P f0f928f3cb6f4a1490e025078b11f5c3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:12:02.796052 10062 raft_consensus.cc:482] T 00000000000000000000000000000000 P f0f928f3cb6f4a1490e025078b11f5c3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:12:02.796094 10062 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f0f928f3cb6f4a1490e025078b11f5c3 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:12:02.796787 10062 raft_consensus.cc:504] T 00000000000000000000000000000000 P f0f928f3cb6f4a1490e025078b11f5c3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f0f928f3cb6f4a1490e025078b11f5c3" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 34011 } }
I20260807 15:12:02.796886 10062 leader_election.cc:302] T 00000000000000000000000000000000 P f0f928f3cb6f4a1490e025078b11f5c3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f0f928f3cb6f4a1490e025078b11f5c3; no voters:
I20260807 15:12:02.797056 10062 leader_election.cc:288] T 00000000000000000000000000000000 P f0f928f3cb6f4a1490e025078b11f5c3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 15:12:02.797084 10068 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f0f928f3cb6f4a1490e025078b11f5c3 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:12:02.797266 10068 raft_consensus.cc:686] T 00000000000000000000000000000000 P f0f928f3cb6f4a1490e025078b11f5c3 [term 1 LEADER]: Becoming Leader. State: Replica: f0f928f3cb6f4a1490e025078b11f5c3, State: Running, Role: LEADER
I20260807 15:12:02.797461 10068 consensus_queue.cc:237] T 00000000000000000000000000000000 P f0f928f3cb6f4a1490e025078b11f5c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f0f928f3cb6f4a1490e025078b11f5c3" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 34011 } }
I20260807 15:12:02.797770 10062 sys_catalog.cc:564] T 00000000000000000000000000000000 P f0f928f3cb6f4a1490e025078b11f5c3 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:12:02.797833 10070 sys_catalog.cc:455] T 00000000000000000000000000000000 P f0f928f3cb6f4a1490e025078b11f5c3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f0f928f3cb6f4a1490e025078b11f5c3. Latest consensus state: current_term: 1 leader_uuid: "f0f928f3cb6f4a1490e025078b11f5c3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f0f928f3cb6f4a1490e025078b11f5c3" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 34011 } } }
I20260807 15:12:02.797914 10070 sys_catalog.cc:458] T 00000000000000000000000000000000 P f0f928f3cb6f4a1490e025078b11f5c3 [sys.catalog]: This master's current role is: LEADER
I20260807 15:12:02.798153 10069 sys_catalog.cc:455] T 00000000000000000000000000000000 P f0f928f3cb6f4a1490e025078b11f5c3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f0f928f3cb6f4a1490e025078b11f5c3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f0f928f3cb6f4a1490e025078b11f5c3" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 34011 } } }
I20260807 15:12:02.798462 10069 sys_catalog.cc:458] T 00000000000000000000000000000000 P f0f928f3cb6f4a1490e025078b11f5c3 [sys.catalog]: This master's current role is: LEADER
I20260807 15:12:02.798600 10076 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:12:02.799134 10076 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 07 15:12:02 dist-test-slave-42z9 krb5kdc[9940](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115522, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.194.62@KRBTEST.COM
I20260807 15:12:02.800803 10076 catalog_manager.cc:1347] Generated new cluster ID: bf72cd3acf864f0ca94352d417008e84
I20260807 15:12:02.800858 10076 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:12:02.803959 10066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:02.792534 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36990 (local address 127.29.194.62:34011)
0807 15:12:02.792740 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0807 15:12:02.792749 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 15:12:02.792782 (+ 33us) server_negotiation.cc:407] Connection header received
0807 15:12:02.793329 (+ 547us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:12:02.793337 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:12:02.793406 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:12:02.793577 (+ 171us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:12:02.794394 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:02.795432 (+ 1038us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:02.796490 (+ 1058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:02.796963 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:02.797078 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:12:02.799863 (+ 2785us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:12:02.799868 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:12:02.799868 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:12:02.799882 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:12:02.801863 (+ 1981us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:12:02.802554 (+ 691us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:12:02.802557 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:12:02.802558 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:12:02.802686 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:12:02.802931 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:12:02.802932 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:12:02.802932 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:12:02.803256 (+ 324us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:12:02.803437 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0807 15:12:02.803586 (+ 149us) server_negotiation.cc:299] Negotiation successful
0807 15:12:02.803745 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":144,"thread_start_us":99,"threads_started":1}
I20260807 15:12:02.806565 10076 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:12:02.807301 10076 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:12:02.810869 10076 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f0f928f3cb6f4a1490e025078b11f5c3: Generated new TSK 0
I20260807 15:12:02.811076 10076 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.29.194.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.194.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.194.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.1.keytab.
Entry for principal kudu/127.29.194.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.1.keytab.
I20260807 15:12:02.849500 30472 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7fZSIL/build/release/bin/kudu
/tmp/dist-test-task7fZSIL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115480851507-30472-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115480851507-30472-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115480851507-30472-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115480851507-30472-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.194.1:0
--local_ip_for_outbound_sockets=127.29.194.1
--webserver_interface=127.29.194.1
--webserver_port=0
--tserver_master_addrs=127.29.194.62:34011
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.1.keytab
--principal=kudu/127.29.194.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.194.20:44991
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115480851507-30472-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115480851507-30472-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115480851507-30472-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:12:02.938978 10098 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:12:02.939201 10098 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:12:02.939226 10098 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:12:02.940513 10098 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:12:02.940568 10098 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:12:02.940598 10098 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.194.1
I20260807 15:12:02.942207 10098 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.194.20:44991
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115480851507-30472-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115480851507-30472-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.1.keytab
--principal=kudu/127.29.194.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.194.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115480851507-30472-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.194.1
--webserver_port=0
--tserver_master_addrs=127.29.194.62:34011
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10098
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.194.1
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115480851507-30472-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:12:02.942466 10098 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:12:02.942698 10098 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:12:02.948084 10104 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:12:02.948098 10109 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:12:02.948107 10103 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:12:02.948400 10098 server_base.cc:1034] running on GCE node
I20260807 15:12:02.948645 10098 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:12:02.948910 10098 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:12:02.950093 10098 hybrid_clock.cc:648] HybridClock initialized: now 1786115522950076 us; error 39 us; skew 500 ppm
Aug 07 15:12:02 dist-test-slave-42z9 krb5kdc[9940](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115522, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.194.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:12:02.953666 10098 init.cc:377] Logged in from keytab as kudu/127.29.194.1@KRBTEST.COM (short username kudu)
I20260807 15:12:02.954195 10098 webserver.cc:460] Webserver started at http://127.29.194.1:40163/ using document root <none> and password file <none>
I20260807 15:12:02.954418 10098 fs_manager.cc:359] Metadata directory not provided
I20260807 15:12:02.954479 10098 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:12:02.954615 10098 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:12:02.955729 10098 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115480851507-30472-0/minicluster-data/ts-0/data/instance:
uuid: "79b163a9dfb44f429721d0abd75c896d"
format_stamp: "Formatted at 2026-08-07 15:12:02 on dist-test-slave-42z9"
server_key: "f6141420c886fbe908dc723fbb2a6e84"
server_key_iv: "9cc8a7675d2a319704ace48a929a7e4f"
server_key_version: "encryptionkey@0"
I20260807 15:12:02.956106 10098 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115480851507-30472-0/minicluster-data/ts-0/wal/instance:
uuid: "79b163a9dfb44f429721d0abd75c896d"
format_stamp: "Formatted at 2026-08-07 15:12:02 on dist-test-slave-42z9"
server_key: "f6141420c886fbe908dc723fbb2a6e84"
server_key_iv: "9cc8a7675d2a319704ace48a929a7e4f"
server_key_version: "encryptionkey@0"
I20260807 15:12:02.957600 10098 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 15:12:02.958581 10117 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:12:02.958847 10098 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:12:02.958951 10098 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115480851507-30472-0/minicluster-data/ts-0/data,/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115480851507-30472-0/minicluster-data/ts-0/wal
uuid: "79b163a9dfb44f429721d0abd75c896d"
format_stamp: "Formatted at 2026-08-07 15:12:02 on dist-test-slave-42z9"
server_key: "f6141420c886fbe908dc723fbb2a6e84"
server_key_iv: "9cc8a7675d2a319704ace48a929a7e4f"
server_key_version: "encryptionkey@0"
I20260807 15:12:02.959036 10098 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115480851507-30472-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115480851507-30472-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115480851507-30472-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:12:02.971819 10098 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:12:02.974303 10098 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:12:02.974471 10098 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:12:02.974715 10098 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:12:02.975032 10098 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:12:02.975068 10098 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:12:02.975099 10098 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:12:02.975123 10098 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:12:02.980924 10098 rpc_server.cc:307] RPC server started. Bound to: 127.29.194.1:34595
I20260807 15:12:02.981005 10291 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.194.1:34595 every 8 connection(s)
I20260807 15:12:02.981321 10098 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115480851507-30472-0/minicluster-data/ts-0/data/info.pb
I20260807 15:12:02.986636 30472 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7fZSIL/build/release/bin/kudu as pid 10098
I20260807 15:12:02.986768 30472 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115480851507-30472-0/minicluster-data/ts-0/wal/instance
I20260807 15:12:02.986970 30472 external_mini_cluster.cc:1428] Setting key dc3e3e0ae2acd1c322f65815910044ae
Aug 07 15:12:02 dist-test-slave-42z9 krb5kdc[9940](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115522, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.194.1@KRBTEST.COM for kudu/127.29.194.62@KRBTEST.COM
I20260807 15:12:02.991184 10066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:02.982106 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.194.1:34913 (local address 127.29.194.62:34011)
0807 15:12:02.982161 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0807 15:12:02.982163 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:12:02.982380 (+ 217us) server_negotiation.cc:407] Connection header received
0807 15:12:02.982872 (+ 492us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:12:02.982874 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:12:02.982887 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:12:02.982949 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:12:02.983893 (+ 944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:02.984403 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:02.985401 (+ 998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:02.985675 (+ 274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:02.985734 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:12:02.988475 (+ 2741us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:12:02.988477 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:12:02.988477 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:12:02.988485 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:12:02.989805 (+ 1320us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:12:02.990218 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:12:02.990220 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:12:02.990220 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:12:02.990277 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:12:02.990517 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:12:02.990518 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:12:02.990518 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:12:02.990637 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:12:02.990697 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0807 15:12:02.990971 (+ 274us) server_negotiation.cc:299] Negotiation successful
0807 15:12:02.991089 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260807 15:12:02.991353 10294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:02.982061 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:34011 (local address 127.29.194.1:34913)
0807 15:12:02.982244 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0807 15:12:02.982281 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0807 15:12:02.982778 (+ 497us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:12:02.983021 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:12:02.983025 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:12:02.983595 (+ 570us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:12:02.983797 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:12:02.983802 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:12:02.984493 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:12:02.984494 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:12:02.985287 (+ 793us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:12:02.985290 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:12:02.985755 (+ 465us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:12:02.985756 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:12:02.985886 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:12:02.986594 (+ 708us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:12:02.986605 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:12:02.988375 (+ 1770us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:12:02.989886 (+ 1511us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:12:02.989895 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:12:02.989897 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:12:02.990145 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:12:02.990337 (+ 192us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:12:02.990338 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:12:02.990338 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:12:02.990452 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:12:02.990695 (+ 243us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:12:02.990703 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:12:02.990847 (+ 144us) client_negotiation.cc:769] Sending connection context
0807 15:12:02.990941 (+ 94us) client_negotiation.cc:240] Negotiation successful
0807 15:12:02.991122 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":92,"threads_started":1}
I20260807 15:12:02.991860 10292 heartbeater.cc:347] Connected to a master server at 127.29.194.62:34011
I20260807 15:12:02.991966 10292 heartbeater.cc:464] Registering TS with master...
I20260807 15:12:02.992157 10292 heartbeater.cc:511] Master 127.29.194.62:34011 requested a full tablet report, sending...
I20260807 15:12:02.992707 10001 ts_manager.cc:194] Registered new tserver with Master: 79b163a9dfb44f429721d0abd75c896d (127.29.194.1:34595)
I20260807 15:12:02.993309 10001 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.194.1@KRBTEST.COM'} at 127.29.194.1:34913
WARNING: no policy specified for kudu/127.29.194.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.194.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.194.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.2.keytab.
Entry for principal kudu/127.29.194.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.2.keytab.
I20260807 15:12:03.017729 30472 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7fZSIL/build/release/bin/kudu
/tmp/dist-test-task7fZSIL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115480851507-30472-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115480851507-30472-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115480851507-30472-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115480851507-30472-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.194.2:0
--local_ip_for_outbound_sockets=127.29.194.2
--webserver_interface=127.29.194.2
--webserver_port=0
--tserver_master_addrs=127.29.194.62:34011
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.2.keytab
--principal=kudu/127.29.194.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.194.20:44991
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115480851507-30472-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115480851507-30472-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115480851507-30472-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:12:03.106805 10298 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:12:03.107064 10298 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:12:03.107092 10298 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:12:03.108357 10298 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:12:03.108413 10298 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:12:03.108441 10298 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.194.2
I20260807 15:12:03.110078 10298 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.194.20:44991
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115480851507-30472-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115480851507-30472-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.2.keytab
--principal=kudu/127.29.194.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.194.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115480851507-30472-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.194.2
--webserver_port=0
--tserver_master_addrs=127.29.194.62:34011
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10298
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.194.2
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115480851507-30472-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:12:03.110347 10298 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:12:03.110592 10298 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:12:03.115785 10307 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:12:03.115765 10310 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:12:03.115760 10306 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:12:03.115877 10298 server_base.cc:1034] running on GCE node
I20260807 15:12:03.116300 10298 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:12:03.116555 10298 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:12:03.117738 10298 hybrid_clock.cc:648] HybridClock initialized: now 1786115523117701 us; error 47 us; skew 500 ppm
Aug 07 15:12:03 dist-test-slave-42z9 krb5kdc[9940](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115523, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.194.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:12:03.120884 10298 init.cc:377] Logged in from keytab as kudu/127.29.194.2@KRBTEST.COM (short username kudu)
I20260807 15:12:03.121388 10298 webserver.cc:460] Webserver started at http://127.29.194.2:46605/ using document root <none> and password file <none>
I20260807 15:12:03.121623 10298 fs_manager.cc:359] Metadata directory not provided
I20260807 15:12:03.121675 10298 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:12:03.121809 10298 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:12:03.122877 10298 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115480851507-30472-0/minicluster-data/ts-1/data/instance:
uuid: "1c687baca3a14223af5bf9389c1749d4"
format_stamp: "Formatted at 2026-08-07 15:12:03 on dist-test-slave-42z9"
server_key: "4e5096f00981881c77ff8f92473fd4e2"
server_key_iv: "bc3c07607dac0dc2eaebf33cec65cc08"
server_key_version: "encryptionkey@0"
I20260807 15:12:03.123214 10298 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115480851507-30472-0/minicluster-data/ts-1/wal/instance:
uuid: "1c687baca3a14223af5bf9389c1749d4"
format_stamp: "Formatted at 2026-08-07 15:12:03 on dist-test-slave-42z9"
server_key: "4e5096f00981881c77ff8f92473fd4e2"
server_key_iv: "bc3c07607dac0dc2eaebf33cec65cc08"
server_key_version: "encryptionkey@0"
I20260807 15:12:03.124557 10298 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 15:12:03.125614 10318 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:12:03.125829 10298 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:12:03.125928 10298 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115480851507-30472-0/minicluster-data/ts-1/data,/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115480851507-30472-0/minicluster-data/ts-1/wal
uuid: "1c687baca3a14223af5bf9389c1749d4"
format_stamp: "Formatted at 2026-08-07 15:12:03 on dist-test-slave-42z9"
server_key: "4e5096f00981881c77ff8f92473fd4e2"
server_key_iv: "bc3c07607dac0dc2eaebf33cec65cc08"
server_key_version: "encryptionkey@0"
I20260807 15:12:03.125999 10298 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115480851507-30472-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115480851507-30472-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115480851507-30472-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:12:03.140259 10298 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:12:03.142616 10298 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:12:03.142765 10298 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:12:03.143021 10298 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:12:03.143339 10298 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:12:03.143376 10298 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:12:03.143407 10298 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:12:03.143424 10298 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:12:03.151053 10298 rpc_server.cc:307] RPC server started. Bound to: 127.29.194.2:44069
I20260807 15:12:03.151131 10481 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.194.2:44069 every 8 connection(s)
I20260807 15:12:03.151525 10298 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115480851507-30472-0/minicluster-data/ts-1/data/info.pb
I20260807 15:12:03.154731 30472 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7fZSIL/build/release/bin/kudu as pid 10298
I20260807 15:12:03.154841 30472 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115480851507-30472-0/minicluster-data/ts-1/wal/instance
I20260807 15:12:03.155023 30472 external_mini_cluster.cc:1428] Setting key 647abcda23aba2365dd5a5b86d15fec8
Aug 07 15:12:03 dist-test-slave-42z9 krb5kdc[9940](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115523, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.194.2@KRBTEST.COM for kudu/127.29.194.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.29.194.3@KRBTEST.COM; defaulting to no policy
I20260807 15:12:03.165833 10484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:03.154085 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:34011 (local address 127.29.194.2:40907)
0807 15:12:03.154274 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0807 15:12:03.154322 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0807 15:12:03.155015 (+ 693us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:12:03.155428 (+ 413us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:12:03.155432 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:12:03.156210 (+ 778us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:12:03.156510 (+ 300us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:12:03.156517 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:12:03.157588 (+ 1071us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:12:03.157590 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:12:03.158961 (+ 1371us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:12:03.158964 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:12:03.159371 (+ 407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:12:03.159372 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:12:03.159515 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:12:03.160039 (+ 524us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:12:03.160048 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:12:03.162061 (+ 2013us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:12:03.164081 (+ 2020us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:12:03.164093 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:12:03.164096 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:12:03.164400 (+ 304us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:12:03.164629 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:12:03.164629 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:12:03.164630 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:12:03.164779 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:12:03.165088 (+ 309us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:12:03.165095 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:12:03.165239 (+ 144us) client_negotiation.cc:769] Sending connection context
0807 15:12:03.165342 (+ 103us) client_negotiation.cc:240] Negotiation successful
0807 15:12:03.165622 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":93,"threads_started":1}
I20260807 15:12:03.165843 10066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:03.154089 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.194.2:40907 (local address 127.29.194.62:34011)
0807 15:12:03.154136 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0807 15:12:03.154139 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:12:03.154424 (+ 285us) server_negotiation.cc:407] Connection header received
0807 15:12:03.155172 (+ 748us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:12:03.155173 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:12:03.155188 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:12:03.155259 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:12:03.156644 (+ 1385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:03.157461 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:03.159069 (+ 1608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:03.159281 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:03.159355 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:12:03.162227 (+ 2872us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:12:03.162230 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:12:03.162230 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:12:03.162240 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:12:03.163956 (+ 1716us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:12:03.164481 (+ 525us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:12:03.164483 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:12:03.164483 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:12:03.164551 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:12:03.164835 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:12:03.164836 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:12:03.164836 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:12:03.164995 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:12:03.165061 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0807 15:12:03.165502 (+ 441us) server_negotiation.cc:299] Negotiation successful
0807 15:12:03.165768 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260807 15:12:03.166306 10482 heartbeater.cc:347] Connected to a master server at 127.29.194.62:34011
I20260807 15:12:03.166407 10482 heartbeater.cc:464] Registering TS with master...
I20260807 15:12:03.166596 10482 heartbeater.cc:511] Master 127.29.194.62:34011 requested a full tablet report, sending...
I20260807 15:12:03.167171 10001 ts_manager.cc:194] Registered new tserver with Master: 1c687baca3a14223af5bf9389c1749d4 (127.29.194.2:44069)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.194.3@KRBTEST.COM" created.
I20260807 15:12:03.167856 10001 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.194.2@KRBTEST.COM'} at 127.29.194.2:40907
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.194.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.3.keytab.
Entry for principal kudu/127.29.194.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.3.keytab.
I20260807 15:12:03.185429 30472 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7fZSIL/build/release/bin/kudu
/tmp/dist-test-task7fZSIL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115480851507-30472-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115480851507-30472-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115480851507-30472-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115480851507-30472-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.194.3:0
--local_ip_for_outbound_sockets=127.29.194.3
--webserver_interface=127.29.194.3
--webserver_port=0
--tserver_master_addrs=127.29.194.62:34011
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.3.keytab
--principal=kudu/127.29.194.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.194.20:44991
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115480851507-30472-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115480851507-30472-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115480851507-30472-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:12:03.270748 10487 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:12:03.270990 10487 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:12:03.271016 10487 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:12:03.272275 10487 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:12:03.272331 10487 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:12:03.272361 10487 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.194.3
I20260807 15:12:03.273926 10487 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.194.20:44991
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115480851507-30472-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115480851507-30472-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.3.keytab
--principal=kudu/127.29.194.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.194.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115480851507-30472-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.194.3
--webserver_port=0
--tserver_master_addrs=127.29.194.62:34011
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10487
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.194.3
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115480851507-30472-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:12:03.274183 10487 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:12:03.274415 10487 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:12:03.279894 10498 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:12:03.279888 10504 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:12:03.279983 10501 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:12:03.280383 10487 server_base.cc:1034] running on GCE node
I20260807 15:12:03.280606 10487 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:12:03.280881 10487 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:12:03.282029 10487 hybrid_clock.cc:648] HybridClock initialized: now 1786115523282057 us; error 80 us; skew 500 ppm
Aug 07 15:12:03 dist-test-slave-42z9 krb5kdc[9940](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115523, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.194.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:12:03.284591 10487 init.cc:377] Logged in from keytab as kudu/127.29.194.3@KRBTEST.COM (short username kudu)
I20260807 15:12:03.285158 10487 webserver.cc:460] Webserver started at http://127.29.194.3:40087/ using document root <none> and password file <none>
I20260807 15:12:03.285382 10487 fs_manager.cc:359] Metadata directory not provided
I20260807 15:12:03.285430 10487 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:12:03.285593 10487 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:12:03.286720 10487 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115480851507-30472-0/minicluster-data/ts-2/data/instance:
uuid: "30335b7dd66145a482336d6e2af96e95"
format_stamp: "Formatted at 2026-08-07 15:12:03 on dist-test-slave-42z9"
server_key: "297306e26c619b2424a3f28e7faf5a46"
server_key_iv: "4f5e1b04172438d05176c0e1b1d71954"
server_key_version: "encryptionkey@0"
I20260807 15:12:03.287051 10487 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115480851507-30472-0/minicluster-data/ts-2/wal/instance:
uuid: "30335b7dd66145a482336d6e2af96e95"
format_stamp: "Formatted at 2026-08-07 15:12:03 on dist-test-slave-42z9"
server_key: "297306e26c619b2424a3f28e7faf5a46"
server_key_iv: "4f5e1b04172438d05176c0e1b1d71954"
server_key_version: "encryptionkey@0"
I20260807 15:12:03.288535 10487 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260807 15:12:03.290135 10512 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:12:03.290323 10487 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 15:12:03.290411 10487 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115480851507-30472-0/minicluster-data/ts-2/data,/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115480851507-30472-0/minicluster-data/ts-2/wal
uuid: "30335b7dd66145a482336d6e2af96e95"
format_stamp: "Formatted at 2026-08-07 15:12:03 on dist-test-slave-42z9"
server_key: "297306e26c619b2424a3f28e7faf5a46"
server_key_iv: "4f5e1b04172438d05176c0e1b1d71954"
server_key_version: "encryptionkey@0"
I20260807 15:12:03.290480 10487 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115480851507-30472-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115480851507-30472-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115480851507-30472-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:12:03.312495 10487 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:12:03.314571 10487 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:12:03.314689 10487 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:12:03.314901 10487 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:12:03.315223 10487 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:12:03.315258 10487 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:12:03.315281 10487 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:12:03.315294 10487 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:12:03.322552 10487 rpc_server.cc:307] RPC server started. Bound to: 127.29.194.3:39165
I20260807 15:12:03.322643 10689 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.194.3:39165 every 8 connection(s)
I20260807 15:12:03.322947 10487 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115480851507-30472-0/minicluster-data/ts-2/data/info.pb
Aug 07 15:12:03 dist-test-slave-42z9 krb5kdc[9940](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115523, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.194.3@KRBTEST.COM for kudu/127.29.194.62@KRBTEST.COM
I20260807 15:12:03.332123 30472 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7fZSIL/build/release/bin/kudu as pid 10487
I20260807 15:12:03.332226 30472 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115480851507-30472-0/minicluster-data/ts-2/wal/instance
I20260807 15:12:03.332259 10066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:03.323857 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.194.3:56183 (local address 127.29.194.62:34011)
0807 15:12:03.323974 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0807 15:12:03.323976 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:12:03.324115 (+ 139us) server_negotiation.cc:407] Connection header received
0807 15:12:03.324674 (+ 559us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:12:03.324676 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:12:03.324690 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:12:03.324743 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:12:03.325763 (+ 1020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:03.326281 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:03.327257 (+ 976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:03.327448 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:03.327505 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:12:03.329860 (+ 2355us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:12:03.329861 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:12:03.329861 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:12:03.329870 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:12:03.331001 (+ 1131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:12:03.331376 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:12:03.331377 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:12:03.331377 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:12:03.331427 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:12:03.331645 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:12:03.331646 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:12:03.331646 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:12:03.331757 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:12:03.331812 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0807 15:12:03.332088 (+ 276us) server_negotiation.cc:299] Negotiation successful
0807 15:12:03.332170 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260807 15:12:03.332417 30472 external_mini_cluster.cc:1428] Setting key 03592cc8464bb10e0e89d8a45585706c
I20260807 15:12:03.332427 10693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:03.323745 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:34011 (local address 127.29.194.3:56183)
0807 15:12:03.323972 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0807 15:12:03.324026 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0807 15:12:03.324572 (+ 546us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:12:03.324814 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:12:03.324818 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:12:03.325416 (+ 598us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:12:03.325676 (+ 260us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:12:03.325681 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:12:03.326368 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:12:03.326370 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:12:03.327175 (+ 805us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:12:03.327177 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:12:03.327518 (+ 341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:12:03.327519 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:12:03.327645 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:12:03.328174 (+ 529us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:12:03.328183 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:12:03.329741 (+ 1558us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:12:03.331080 (+ 1339us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:12:03.331088 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:12:03.331090 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:12:03.331312 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:12:03.331479 (+ 167us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:12:03.331479 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:12:03.331480 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:12:03.331589 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:12:03.331810 (+ 221us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:12:03.331817 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:12:03.331966 (+ 149us) client_negotiation.cc:769] Sending connection context
0807 15:12:03.332053 (+ 87us) client_negotiation.cc:240] Negotiation successful
0807 15:12:03.332211 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":92,"threads_started":1}
I20260807 15:12:03.332850 10691 heartbeater.cc:347] Connected to a master server at 127.29.194.62:34011
I20260807 15:12:03.332949 10691 heartbeater.cc:464] Registering TS with master...
I20260807 15:12:03.333132 10691 heartbeater.cc:511] Master 127.29.194.62:34011 requested a full tablet report, sending...
I20260807 15:12:03.333523 9997 ts_manager.cc:194] Registered new tserver with Master: 30335b7dd66145a482336d6e2af96e95 (127.29.194.3:39165)
I20260807 15:12:03.333890 9997 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.194.3@KRBTEST.COM'} at 127.29.194.3:56183
I20260807 15:12:03.344393 30472 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.29.194.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.29.194.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.29.194.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115480851507-30472-0/krb5kdc/oryx_127.29.194.1.keytab.
Entry for principal oryx/127.29.194.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115480851507-30472-0/krb5kdc/oryx_127.29.194.1.keytab.
WARNING: no policy specified for oryx/127.29.194.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.29.194.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.29.194.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115480851507-30472-0/krb5kdc/oryx_127.29.194.2.keytab.
Entry for principal oryx/127.29.194.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115480851507-30472-0/krb5kdc/oryx_127.29.194.2.keytab.
WARNING: no policy specified for oryx/127.29.194.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.29.194.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.29.194.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115480851507-30472-0/krb5kdc/oryx_127.29.194.3.keytab.
Entry for principal oryx/127.29.194.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115480851507-30472-0/krb5kdc/oryx_127.29.194.3.keytab.
I20260807 15:12:03.432320 30472 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7fZSIL/build/release/bin/kudu with pid 10098
I20260807 15:12:03.449220 30472 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7fZSIL/build/release/bin/kudu with pid 10298
I20260807 15:12:03.466251 30472 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7fZSIL/build/release/bin/kudu with pid 10487
I20260807 15:12:03.480562 30472 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7fZSIL/build/release/bin/kudu with pid 9959
I20260807 15:12:03.491683 30472 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7fZSIL/build/release/bin/kudu
/tmp/dist-test-task7fZSIL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115480851507-30472-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115480851507-30472-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115480851507-30472-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115480851507-30472-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.194.62:34011
--webserver_interface=127.29.194.62
--webserver_port=39211
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.62.keytab
--principal=kudu/127.29.194.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.194.20:44991
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.194.62:34011
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115480851507-30472-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115480851507-30472-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115480851507-30472-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:12:03.578123 10703 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:12:03.578372 10703 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:12:03.578406 10703 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:12:03.580842 10703 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:12:03.580935 10703 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:12:03.580960 10703 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:12:03.580978 10703 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:12:03.580993 10703 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:12:03.583125 10703 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.194.20:44991
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115480851507-30472-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115480851507-30472-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.194.62:34011
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.62.keytab
--principal=kudu/127.29.194.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.194.62:34011
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115480851507-30472-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.194.62
--webserver_port=39211
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10703
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115480851507-30472-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:12:03.583457 10703 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:12:03.583694 10703 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:12:03.589414 10711 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:12:03.589416 10714 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:12:03.589414 10712 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:12:03.589464 10703 server_base.cc:1034] running on GCE node
I20260807 15:12:03.589920 10703 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:12:03.590224 10703 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:12:03.591383 10703 hybrid_clock.cc:648] HybridClock initialized: now 1786115523591375 us; error 49 us; skew 500 ppm
Aug 07 15:12:03 dist-test-slave-42z9 krb5kdc[9940](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115523, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.194.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:12:03.594272 10703 init.cc:377] Logged in from keytab as kudu/127.29.194.62@KRBTEST.COM (short username kudu)
I20260807 15:12:03.594785 10703 webserver.cc:460] Webserver started at http://127.29.194.62:39211/ using document root <none> and password file <none>
I20260807 15:12:03.594985 10703 fs_manager.cc:359] Metadata directory not provided
I20260807 15:12:03.595044 10703 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:12:03.596576 10703 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:12:03.597463 10722 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:12:03.597638 10703 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:12:03.597734 10703 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115480851507-30472-0/minicluster-data/master-0/data,/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115480851507-30472-0/minicluster-data/master-0/wal
uuid: "f0f928f3cb6f4a1490e025078b11f5c3"
format_stamp: "Formatted at 2026-08-07 15:12:02 on dist-test-slave-42z9"
server_key: "9dee93f8b06be22d17313992eeea50e7"
server_key_iv: "b4a48b6fff5f413dcfdcfb739c5b318f"
server_key_version: "encryptionkey@0"
I20260807 15:12:03.598100 10703 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115480851507-30472-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115480851507-30472-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115480851507-30472-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:12:03.609758 10703 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:12:03.612200 10703 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:12:03.612372 10703 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:12:03.617746 10703 rpc_server.cc:307] RPC server started. Bound to: 127.29.194.62:34011
I20260807 15:12:03.617900 10797 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.194.62:34011 every 8 connection(s)
I20260807 15:12:03.618254 10703 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115480851507-30472-0/minicluster-data/master-0/data/info.pb
I20260807 15:12:03.620026 10798 sys_catalog.cc:263] Verifying existing consensus state
I20260807 15:12:03.620808 10798 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f0f928f3cb6f4a1490e025078b11f5c3: Bootstrap starting.
I20260807 15:12:03.628254 30472 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7fZSIL/build/release/bin/kudu as pid 10703
I20260807 15:12:03.628657 30472 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7fZSIL/build/release/bin/kudu
/tmp/dist-test-task7fZSIL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115480851507-30472-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115480851507-30472-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115480851507-30472-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115480851507-30472-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.194.1:34595
--local_ip_for_outbound_sockets=127.29.194.1
--tserver_master_addrs=127.29.194.62:34011
--webserver_port=40163
--webserver_interface=127.29.194.1
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.1.keytab
--principal=kudu/127.29.194.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.194.20:44991
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.29.194.1
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115480851507-30472-0/krb5kdc/oryx_127.29.194.1.keytab with env {KRB5CCNAME=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115480851507-30472-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115480851507-30472-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115480851507-30472-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 15:12:03.630246 10798 log.cc:824] T 00000000000000000000000000000000 P f0f928f3cb6f4a1490e025078b11f5c3: Log is configured to *not* fsync() on all Append() calls
I20260807 15:12:03.632550 10798 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f0f928f3cb6f4a1490e025078b11f5c3: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 15:12:03.632836 10798 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f0f928f3cb6f4a1490e025078b11f5c3: Bootstrap complete.
I20260807 15:12:03.635145 10798 raft_consensus.cc:348] T 00000000000000000000000000000000 P f0f928f3cb6f4a1490e025078b11f5c3 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f0f928f3cb6f4a1490e025078b11f5c3" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 34011 } }
I20260807 15:12:03.635375 10798 raft_consensus.cc:729] T 00000000000000000000000000000000 P f0f928f3cb6f4a1490e025078b11f5c3 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: f0f928f3cb6f4a1490e025078b11f5c3, State: Initialized, Role: FOLLOWER
I20260807 15:12:03.635519 10798 consensus_queue.cc:260] T 00000000000000000000000000000000 P f0f928f3cb6f4a1490e025078b11f5c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f0f928f3cb6f4a1490e025078b11f5c3" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 34011 } }
I20260807 15:12:03.635586 10798 raft_consensus.cc:388] T 00000000000000000000000000000000 P f0f928f3cb6f4a1490e025078b11f5c3 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:12:03.635633 10798 raft_consensus.cc:482] T 00000000000000000000000000000000 P f0f928f3cb6f4a1490e025078b11f5c3 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:12:03.635677 10798 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f0f928f3cb6f4a1490e025078b11f5c3 [term 1 FOLLOWER]: Advancing to term 2
I20260807 15:12:03.636699 10798 raft_consensus.cc:504] T 00000000000000000000000000000000 P f0f928f3cb6f4a1490e025078b11f5c3 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f0f928f3cb6f4a1490e025078b11f5c3" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 34011 } }
I20260807 15:12:03.636794 10798 leader_election.cc:302] T 00000000000000000000000000000000 P f0f928f3cb6f4a1490e025078b11f5c3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f0f928f3cb6f4a1490e025078b11f5c3; no voters:
I20260807 15:12:03.636976 10798 leader_election.cc:288] T 00000000000000000000000000000000 P f0f928f3cb6f4a1490e025078b11f5c3 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260807 15:12:03.637032 10806 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f0f928f3cb6f4a1490e025078b11f5c3 [term 2 FOLLOWER]: Leader election won for term 2
I20260807 15:12:03.637200 10806 raft_consensus.cc:686] T 00000000000000000000000000000000 P f0f928f3cb6f4a1490e025078b11f5c3 [term 2 LEADER]: Becoming Leader. State: Replica: f0f928f3cb6f4a1490e025078b11f5c3, State: Running, Role: LEADER
I20260807 15:12:03.637274 10798 sys_catalog.cc:564] T 00000000000000000000000000000000 P f0f928f3cb6f4a1490e025078b11f5c3 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:12:03.637349 10806 consensus_queue.cc:237] T 00000000000000000000000000000000 P f0f928f3cb6f4a1490e025078b11f5c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f0f928f3cb6f4a1490e025078b11f5c3" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 34011 } }
I20260807 15:12:03.637769 10806 sys_catalog.cc:455] T 00000000000000000000000000000000 P f0f928f3cb6f4a1490e025078b11f5c3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f0f928f3cb6f4a1490e025078b11f5c3. Latest consensus state: current_term: 2 leader_uuid: "f0f928f3cb6f4a1490e025078b11f5c3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f0f928f3cb6f4a1490e025078b11f5c3" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 34011 } } }
I20260807 15:12:03.637854 10806 sys_catalog.cc:458] T 00000000000000000000000000000000 P f0f928f3cb6f4a1490e025078b11f5c3 [sys.catalog]: This master's current role is: LEADER
I20260807 15:12:03.638130 10809 sys_catalog.cc:455] T 00000000000000000000000000000000 P f0f928f3cb6f4a1490e025078b11f5c3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "f0f928f3cb6f4a1490e025078b11f5c3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f0f928f3cb6f4a1490e025078b11f5c3" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 34011 } } }
I20260807 15:12:03.638216 10809 sys_catalog.cc:458] T 00000000000000000000000000000000 P f0f928f3cb6f4a1490e025078b11f5c3 [sys.catalog]: This master's current role is: LEADER
I20260807 15:12:03.638362 10814 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:12:03.638903 10814 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:12:03.639045 10814 catalog_manager.cc:1259] Loaded cluster ID: bf72cd3acf864f0ca94352d417008e84
I20260807 15:12:03.639130 10814 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:12:03.640810 10814 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:12:03.641006 10814 catalog_manager.cc:5879] T 00000000000000000000000000000000 P f0f928f3cb6f4a1490e025078b11f5c3: Loaded TSK: 0
I20260807 15:12:03.641224 10814 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260807 15:12:03.715240 10802 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:12:03.715466 10802 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:12:03.715492 10802 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:12:03.716727 10802 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:12:03.716777 10802 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:12:03.716804 10802 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.194.1
I20260807 15:12:03.718379 10802 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.194.20:44991
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115480851507-30472-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115480851507-30472-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115480851507-30472-0/krb5kdc/oryx_127.29.194.1.keytab
--principal=oryx/127.29.194.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.194.1:34595
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115480851507-30472-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.194.1
--webserver_port=40163
--tserver_master_addrs=127.29.194.62:34011
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10802
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.194.1
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115480851507-30472-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:12:03.718626 10802 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:12:03.718878 10802 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:12:03.724277 10838 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:12:03.724339 10842 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:12:03.724257 10839 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:12:03.724756 10802 server_base.cc:1034] running on GCE node
I20260807 15:12:03.725035 10802 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:12:03.725288 10802 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:12:03.726475 10802 hybrid_clock.cc:648] HybridClock initialized: now 1786115523726452 us; error 47 us; skew 500 ppm
Aug 07 15:12:03 dist-test-slave-42z9 krb5kdc[9940](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115523, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.194.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:12:03.729118 10802 init.cc:377] Logged in from keytab as oryx/127.29.194.1@KRBTEST.COM (short username oryx)
I20260807 15:12:03.729664 10802 webserver.cc:460] Webserver started at http://127.29.194.1:40163/ using document root <none> and password file <none>
I20260807 15:12:03.729897 10802 fs_manager.cc:359] Metadata directory not provided
I20260807 15:12:03.729965 10802 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:12:03.731415 10802 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 15:12:03.732054 10849 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:12:03.732261 10802 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:12:03.732350 10802 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115480851507-30472-0/minicluster-data/ts-0/data,/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115480851507-30472-0/minicluster-data/ts-0/wal
uuid: "79b163a9dfb44f429721d0abd75c896d"
format_stamp: "Formatted at 2026-08-07 15:12:02 on dist-test-slave-42z9"
server_key: "f6141420c886fbe908dc723fbb2a6e84"
server_key_iv: "9cc8a7675d2a319704ace48a929a7e4f"
server_key_version: "encryptionkey@0"
I20260807 15:12:03.732719 10802 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115480851507-30472-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115480851507-30472-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115480851507-30472-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:12:03.753152 10802 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:12:03.755451 10802 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:12:03.755594 10802 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:12:03.755837 10802 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:12:03.756189 10802 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:12:03.756227 10802 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:12:03.756250 10802 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:12:03.756274 10802 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:12:03.763155 10802 rpc_server.cc:307] RPC server started. Bound to: 127.29.194.1:34595
I20260807 15:12:03.763232 11015 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.194.1:34595 every 8 connection(s)
I20260807 15:12:03.763592 10802 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115480851507-30472-0/minicluster-data/ts-0/data/info.pb
I20260807 15:12:03.765429 30472 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7fZSIL/build/release/bin/kudu as pid 10802
I20260807 15:12:03.765805 30472 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7fZSIL/build/release/bin/kudu
/tmp/dist-test-task7fZSIL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115480851507-30472-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115480851507-30472-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115480851507-30472-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115480851507-30472-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.194.2:44069
--local_ip_for_outbound_sockets=127.29.194.2
--tserver_master_addrs=127.29.194.62:34011
--webserver_port=46605
--webserver_interface=127.29.194.2
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.2.keytab
--principal=kudu/127.29.194.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.194.20:44991
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.29.194.2
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115480851507-30472-0/krb5kdc/oryx_127.29.194.2.keytab with env {KRB5CCNAME=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115480851507-30472-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115480851507-30472-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115480851507-30472-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 15:12:03 dist-test-slave-42z9 krb5kdc[9940](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.194.1@KRBTEST.COM for oryx/127.29.194.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 15:12:03 dist-test-slave-42z9 krb5kdc[9940](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.194.1@KRBTEST.COM for oryx/127.29.194.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 15:12:03.772486 11019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:03.764278 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:34011 (local address 127.29.194.1:35465)
0807 15:12:03.764461 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0807 15:12:03.764511 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0807 15:12:03.765113 (+ 602us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:12:03.765706 (+ 593us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:12:03.765710 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:12:03.766457 (+ 747us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:12:03.766723 (+ 266us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:12:03.766728 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:12:03.768047 (+ 1319us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:12:03.768048 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:12:03.768838 (+ 790us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:12:03.768841 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:12:03.769459 (+ 618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:12:03.769461 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:12:03.769588 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:12:03.770106 (+ 518us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:12:03.770115 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:12:03.772269 (+ 2154us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.194.62:34011: Server oryx/127.29.194.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":83,"threads_started":1}
W20260807 15:12:03.772903 11020 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:12:03.764383 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.194.1:35465 (local address 127.29.194.62:34011)
0807 15:12:03.764552 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0807 15:12:03.764561 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 15:12:03.764597 (+ 36us) server_negotiation.cc:407] Connection header received
0807 15:12:03.765404 (+ 807us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:12:03.765412 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:12:03.765508 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:12:03.765657 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:12:03.766863 (+ 1206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:03.767934 (+ 1071us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:03.768940 (+ 1006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:03.769335 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:03.769459 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:12:03.772439 (+ 2980us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.194.1:35465: BlockingRecv error: recv got EOF from 127.29.194.1:35465 (error 108)
Metrics: {"server-negotiator.queue_time_us":120,"thread_start_us":84,"threads_started":1}
W20260807 15:12:03.772902 11017 heartbeater.cc:644] Failed to heartbeat to 127.29.194.62:34011 (0 consecutive failures): Not authorized: Failed to ping master at 127.29.194.62:34011: Client connection negotiation failed: client connection to 127.29.194.62:34011: Server oryx/127.29.194.62@KRBTEST.COM not found in Kerberos database
Aug 07 15:12:03 dist-test-slave-42z9 krb5kdc[9940](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.194.1@KRBTEST.COM for oryx/127.29.194.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 15:12:03 dist-test-slave-42z9 krb5kdc[9940](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.194.1@KRBTEST.COM for oryx/127.29.194.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 15:12:03.777509 11019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:03.773119 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:34011 (local address 127.29.194.1:38147)
0807 15:12:03.773164 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0807 15:12:03.773175 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 15:12:03.773214 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:12:03.773486 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:12:03.773487 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:12:03.773610 (+ 123us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:12:03.773660 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:12:03.773662 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:12:03.774273 (+ 611us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:12:03.774274 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:12:03.774765 (+ 491us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:12:03.774767 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:12:03.775231 (+ 464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:12:03.775233 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:12:03.775314 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:12:03.775738 (+ 424us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:12:03.775743 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:12:03.777414 (+ 1671us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.194.62:34011: Server oryx/127.29.194.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":29}
W20260807 15:12:03.777632 11020 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:12:03.773200 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.194.1:38147 (local address 127.29.194.62:34011)
0807 15:12:03.773337 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0807 15:12:03.773339 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:12:03.773348 (+ 9us) server_negotiation.cc:407] Connection header received
0807 15:12:03.773382 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:12:03.773382 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:12:03.773394 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:12:03.773478 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:12:03.773729 (+ 251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:03.774203 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:03.774838 (+ 635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:03.775129 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:03.775219 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:12:03.777565 (+ 2346us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.194.1:38147: BlockingRecv error: recv got EOF from 127.29.194.1:38147 (error 108)
Metrics: {"server-negotiator.queue_time_us":120,"thread_start_us":108,"threads_started":1}
Aug 07 15:12:03 dist-test-slave-42z9 krb5kdc[9940](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.194.1@KRBTEST.COM for oryx/127.29.194.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 15:12:03 dist-test-slave-42z9 krb5kdc[9940](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.194.1@KRBTEST.COM for oryx/127.29.194.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 15:12:03.783682 11019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:03.777806 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:34011 (local address 127.29.194.1:52025)
0807 15:12:03.777849 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0807 15:12:03.777859 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 15:12:03.777893 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:12:03.778080 (+ 187us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:12:03.778081 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:12:03.778187 (+ 106us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:12:03.778222 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:12:03.778223 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:12:03.779219 (+ 996us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:12:03.779221 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:12:03.779847 (+ 626us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:12:03.779850 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:12:03.780413 (+ 563us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:12:03.780414 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:12:03.780481 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:12:03.781168 (+ 687us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:12:03.781174 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:12:03.783609 (+ 2435us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.194.62:34011: Server oryx/127.29.194.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":28}
W20260807 15:12:03.783797 11020 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:12:03.777898 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.194.1:52025 (local address 127.29.194.62:34011)
0807 15:12:03.777960 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0807 15:12:03.777962 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:12:03.777970 (+ 8us) server_negotiation.cc:407] Connection header received
0807 15:12:03.777993 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:12:03.777994 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:12:03.778004 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:12:03.778058 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:12:03.778278 (+ 220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:03.779039 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:03.779985 (+ 946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:03.780298 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:03.780376 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:12:03.783729 (+ 3353us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.194.1:52025: BlockingRecv error: recv got EOF from 127.29.194.1:52025 (error 108)
Metrics: {"server-negotiator.queue_time_us":33}
W20260807 15:12:03.784021 11017 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260807 15:12:03.854504 11021 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:12:03.854748 11021 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:12:03.854782 11021 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:12:03.856305 11021 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:12:03.856385 11021 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:12:03.856431 11021 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.194.2
I20260807 15:12:03.858107 11021 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.194.20:44991
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115480851507-30472-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115480851507-30472-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115480851507-30472-0/krb5kdc/oryx_127.29.194.2.keytab
--principal=oryx/127.29.194.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.194.2:44069
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115480851507-30472-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.194.2
--webserver_port=46605
--tserver_master_addrs=127.29.194.62:34011
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11021
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.194.2
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115480851507-30472-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:12:03.858376 11021 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:12:03.858618 11021 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:12:03.863713 11031 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:12:03.863667 11034 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:12:03.863677 11029 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:12:03.863831 11021 server_base.cc:1034] running on GCE node
I20260807 15:12:03.864154 11021 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:12:03.864389 11021 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:12:03.865523 11021 hybrid_clock.cc:648] HybridClock initialized: now 1786115523865510 us; error 30 us; skew 500 ppm
Aug 07 15:12:03 dist-test-slave-42z9 krb5kdc[9940](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115523, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.194.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:12:03.868342 11021 init.cc:377] Logged in from keytab as oryx/127.29.194.2@KRBTEST.COM (short username oryx)
I20260807 15:12:03.868817 11021 webserver.cc:460] Webserver started at http://127.29.194.2:46605/ using document root <none> and password file <none>
I20260807 15:12:03.869061 11021 fs_manager.cc:359] Metadata directory not provided
I20260807 15:12:03.869131 11021 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:12:03.870606 11021 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:12:03.871259 11044 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:12:03.871522 11021 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:12:03.871614 11021 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115480851507-30472-0/minicluster-data/ts-1/data,/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115480851507-30472-0/minicluster-data/ts-1/wal
uuid: "1c687baca3a14223af5bf9389c1749d4"
format_stamp: "Formatted at 2026-08-07 15:12:03 on dist-test-slave-42z9"
server_key: "4e5096f00981881c77ff8f92473fd4e2"
server_key_iv: "bc3c07607dac0dc2eaebf33cec65cc08"
server_key_version: "encryptionkey@0"
I20260807 15:12:03.872063 11021 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115480851507-30472-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115480851507-30472-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115480851507-30472-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:12:03.893556 11021 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:12:03.895825 11021 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:12:03.895988 11021 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:12:03.896263 11021 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:12:03.896690 11021 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:12:03.896730 11021 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:12:03.896761 11021 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:12:03.896785 11021 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:12:03.903688 11021 rpc_server.cc:307] RPC server started. Bound to: 127.29.194.2:44069
I20260807 15:12:03.903807 11207 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.194.2:44069 every 8 connection(s)
I20260807 15:12:03.904129 11021 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115480851507-30472-0/minicluster-data/ts-1/data/info.pb
Aug 07 15:12:03 dist-test-slave-42z9 krb5kdc[9940](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.194.2@KRBTEST.COM for oryx/127.29.194.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 15:12:03 dist-test-slave-42z9 krb5kdc[9940](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.194.2@KRBTEST.COM for oryx/127.29.194.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 15:12:03.912107 11214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:03.904865 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:34011 (local address 127.29.194.2:56539)
0807 15:12:03.905034 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0807 15:12:03.905078 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0807 15:12:03.905614 (+ 536us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:12:03.906000 (+ 386us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:12:03.906004 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:12:03.906611 (+ 607us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:12:03.906821 (+ 210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:12:03.906825 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:12:03.907592 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:12:03.907593 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:12:03.908401 (+ 808us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:12:03.908403 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:12:03.908784 (+ 381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:12:03.908784 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:12:03.908885 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:12:03.909362 (+ 477us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:12:03.909371 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:12:03.911946 (+ 2575us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.194.62:34011: Server oryx/127.29.194.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":72,"threads_started":1}
W20260807 15:12:03.912163 11020 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:12:03.904879 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.194.2:56539 (local address 127.29.194.62:34011)
0807 15:12:03.904934 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0807 15:12:03.904936 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:12:03.905163 (+ 227us) server_negotiation.cc:407] Connection header received
0807 15:12:03.905793 (+ 630us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:12:03.905796 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:12:03.905821 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:12:03.905929 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:12:03.906975 (+ 1046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:03.907489 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:03.908508 (+ 1019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:03.908708 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:03.908774 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:12:03.912114 (+ 3340us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.194.2:56539: BlockingRecv error: recv got EOF from 127.29.194.2:56539 (error 108)
Metrics: {"server-negotiator.queue_time_us":28}
W20260807 15:12:03.912393 11209 heartbeater.cc:644] Failed to heartbeat to 127.29.194.62:34011 (0 consecutive failures): Not authorized: Failed to ping master at 127.29.194.62:34011: Client connection negotiation failed: client connection to 127.29.194.62:34011: Server oryx/127.29.194.62@KRBTEST.COM not found in Kerberos database
I20260807 15:12:03.913280 30472 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7fZSIL/build/release/bin/kudu as pid 11021
I20260807 15:12:03.913589 30472 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7fZSIL/build/release/bin/kudu
/tmp/dist-test-task7fZSIL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115480851507-30472-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115480851507-30472-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115480851507-30472-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115480851507-30472-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.194.3:39165
--local_ip_for_outbound_sockets=127.29.194.3
--tserver_master_addrs=127.29.194.62:34011
--webserver_port=40087
--webserver_interface=127.29.194.3
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.3.keytab
--principal=kudu/127.29.194.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.194.20:44991
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.29.194.3
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115480851507-30472-0/krb5kdc/oryx_127.29.194.3.keytab with env {KRB5CCNAME=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115480851507-30472-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115480851507-30472-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115480851507-30472-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 15:12:03 dist-test-slave-42z9 krb5kdc[9940](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.194.2@KRBTEST.COM for oryx/127.29.194.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 15:12:03 dist-test-slave-42z9 krb5kdc[9940](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.194.2@KRBTEST.COM for oryx/127.29.194.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 15:12:03.917026 11214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:03.912624 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:34011 (local address 127.29.194.2:58817)
0807 15:12:03.912672 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0807 15:12:03.912683 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 15:12:03.912732 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:12:03.912834 (+ 102us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:12:03.912834 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:12:03.912951 (+ 117us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:12:03.913002 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:12:03.913004 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:12:03.913649 (+ 645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:12:03.913650 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:12:03.914295 (+ 645us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:12:03.914299 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:12:03.914665 (+ 366us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:12:03.914666 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:12:03.914725 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:12:03.915107 (+ 382us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:12:03.915111 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:12:03.916975 (+ 1864us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.194.62:34011: Server oryx/127.29.194.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":28}
W20260807 15:12:03.917224 11020 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:12:03.912674 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.194.2:58817 (local address 127.29.194.62:34011)
0807 15:12:03.912723 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0807 15:12:03.912725 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:12:03.912734 (+ 9us) server_negotiation.cc:407] Connection header received
0807 15:12:03.912778 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:12:03.912779 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:12:03.912787 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:12:03.912830 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:12:03.913048 (+ 218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:03.913576 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:03.914406 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:03.914595 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:03.914656 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:12:03.917167 (+ 2511us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.194.2:58817: BlockingRecv error: recv got EOF from 127.29.194.2:58817 (error 108)
Metrics: {"server-negotiator.queue_time_us":28}
Aug 07 15:12:03 dist-test-slave-42z9 krb5kdc[9940](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.194.2@KRBTEST.COM for oryx/127.29.194.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 15:12:03 dist-test-slave-42z9 krb5kdc[9940](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.194.2@KRBTEST.COM for oryx/127.29.194.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 15:12:03.922040 11214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:03.917331 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:34011 (local address 127.29.194.2:49993)
0807 15:12:03.917412 (+ 81us) negotiation.cc:107] Waiting for socket to connect
0807 15:12:03.917421 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0807 15:12:03.917482 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:12:03.917612 (+ 130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:12:03.917613 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:12:03.917717 (+ 104us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:12:03.917745 (+ 28us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:12:03.917747 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:12:03.918613 (+ 866us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:12:03.918614 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:12:03.919112 (+ 498us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:12:03.919114 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:12:03.919797 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:12:03.919798 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:12:03.919844 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:12:03.920260 (+ 416us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:12:03.920265 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:12:03.921983 (+ 1718us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.194.62:34011: Server oryx/127.29.194.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":66}
W20260807 15:12:03.922178 11020 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:12:03.917412 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.194.2:49993 (local address 127.29.194.62:34011)
0807 15:12:03.917495 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0807 15:12:03.917497 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:12:03.917506 (+ 9us) server_negotiation.cc:407] Connection header received
0807 15:12:03.917530 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:12:03.917531 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:12:03.917539 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:12:03.917610 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:12:03.917813 (+ 203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:03.918519 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:03.919282 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:03.919654 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:03.919794 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:12:03.922117 (+ 2323us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.194.2:49993: BlockingRecv error: recv got EOF from 127.29.194.2:49993 (error 108)
Metrics: {"server-negotiator.queue_time_us":15}
W20260807 15:12:03.922168 11209 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260807 15:12:04.018818 11215 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:12:04.019063 11215 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:12:04.019089 11215 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:12:04.020359 11215 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:12:04.020411 11215 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:12:04.020440 11215 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.194.3
I20260807 15:12:04.022024 11215 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.194.20:44991
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115480851507-30472-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115480851507-30472-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115480851507-30472-0/krb5kdc/oryx_127.29.194.3.keytab
--principal=oryx/127.29.194.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.194.3:39165
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115480851507-30472-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.194.3
--webserver_port=40087
--tserver_master_addrs=127.29.194.62:34011
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11215
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.194.3
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115480851507-30472-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:12:04.022256 11215 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:12:04.022486 11215 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:12:04.027841 11225 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:12:04.027853 11230 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:12:04.027841 11228 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:12:04.028012 11215 server_base.cc:1034] running on GCE node
I20260807 15:12:04.028317 11215 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:12:04.028581 11215 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:12:04.029891 11215 hybrid_clock.cc:648] HybridClock initialized: now 1786115524029773 us; error 34 us; skew 500 ppm
Aug 07 15:12:04 dist-test-slave-42z9 krb5kdc[9940](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115524, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.194.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:12:04.032724 11215 init.cc:377] Logged in from keytab as oryx/127.29.194.3@KRBTEST.COM (short username oryx)
I20260807 15:12:04.033221 11215 webserver.cc:460] Webserver started at http://127.29.194.3:40087/ using document root <none> and password file <none>
I20260807 15:12:04.033433 11215 fs_manager.cc:359] Metadata directory not provided
I20260807 15:12:04.033558 11215 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:12:04.035141 11215 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:12:04.036120 11243 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:12:04.036365 11215 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:12:04.036469 11215 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115480851507-30472-0/minicluster-data/ts-2/data,/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115480851507-30472-0/minicluster-data/ts-2/wal
uuid: "30335b7dd66145a482336d6e2af96e95"
format_stamp: "Formatted at 2026-08-07 15:12:03 on dist-test-slave-42z9"
server_key: "297306e26c619b2424a3f28e7faf5a46"
server_key_iv: "4f5e1b04172438d05176c0e1b1d71954"
server_key_version: "encryptionkey@0"
I20260807 15:12:04.036849 11215 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115480851507-30472-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115480851507-30472-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115480851507-30472-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:12:04.052727 11215 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:12:04.055071 11215 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:12:04.055249 11215 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:12:04.055511 11215 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:12:04.055857 11215 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:12:04.055893 11215 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:12:04.055924 11215 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:12:04.055943 11215 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:12:04.062804 11215 rpc_server.cc:307] RPC server started. Bound to: 127.29.194.3:39165
I20260807 15:12:04.062907 11404 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.194.3:39165 every 8 connection(s)
I20260807 15:12:04.063241 11215 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115480851507-30472-0/minicluster-data/ts-2/data/info.pb
Aug 07 15:12:04 dist-test-slave-42z9 krb5kdc[9940](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.194.3@KRBTEST.COM for oryx/127.29.194.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 15:12:04.070904 30472 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7fZSIL/build/release/bin/kudu as pid 11215
Aug 07 15:12:04 dist-test-slave-42z9 krb5kdc[9940](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.194.3@KRBTEST.COM for oryx/127.29.194.62@KRBTEST.COM, Server not found in Kerberos database
W20260807 15:12:04.072507 11020 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:12:04.064236 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.194.3:46977 (local address 127.29.194.62:34011)
0807 15:12:04.064339 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0807 15:12:04.064341 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:12:04.064549 (+ 208us) server_negotiation.cc:407] Connection header received
0807 15:12:04.065084 (+ 535us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:12:04.065086 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:12:04.065100 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:12:04.065156 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:12:04.066249 (+ 1093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:04.066745 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:04.068034 (+ 1289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:04.068230 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:04.068288 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:12:04.072453 (+ 4165us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.194.3:46977: BlockingRecv error: recv got EOF from 127.29.194.3:46977 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
I20260807 15:12:04.072522 11412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:04.064200 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:34011 (local address 127.29.194.3:46977)
0807 15:12:04.064412 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0807 15:12:04.064457 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0807 15:12:04.064969 (+ 512us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:12:04.065250 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:12:04.065255 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:12:04.065925 (+ 670us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:12:04.066134 (+ 209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:12:04.066140 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:12:04.066835 (+ 695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:12:04.066836 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:12:04.067932 (+ 1096us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:12:04.067936 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:12:04.068304 (+ 368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:12:04.068305 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:12:04.068471 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:12:04.069131 (+ 660us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:12:04.069146 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:12:04.072283 (+ 3137us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.194.62:34011: Server oryx/127.29.194.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":81,"threads_started":1}
W20260807 15:12:04.072827 11408 heartbeater.cc:644] Failed to heartbeat to 127.29.194.62:34011 (0 consecutive failures): Not authorized: Failed to ping master at 127.29.194.62:34011: Client connection negotiation failed: client connection to 127.29.194.62:34011: Server oryx/127.29.194.62@KRBTEST.COM not found in Kerberos database
Aug 07 15:12:04 dist-test-slave-42z9 krb5kdc[9940](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.194.3@KRBTEST.COM for oryx/127.29.194.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 15:12:04 dist-test-slave-42z9 krb5kdc[9940](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.194.3@KRBTEST.COM for oryx/127.29.194.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 15:12:04.077580 11412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:04.073029 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:34011 (local address 127.29.194.3:41819)
0807 15:12:04.073072 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0807 15:12:04.073082 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 15:12:04.073116 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:12:04.073233 (+ 117us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:12:04.073233 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:12:04.073378 (+ 145us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:12:04.073426 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:12:04.073428 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:12:04.074073 (+ 645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:12:04.074074 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:12:04.074625 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:12:04.074628 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:12:04.075036 (+ 408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:12:04.075037 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:12:04.075091 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:12:04.075427 (+ 336us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:12:04.075431 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:12:04.077504 (+ 2073us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.194.62:34011: Server oryx/127.29.194.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":25}
W20260807 15:12:04.077637 11020 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:12:04.073089 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.194.3:41819 (local address 127.29.194.62:34011)
0807 15:12:04.073144 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0807 15:12:04.073146 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:12:04.073154 (+ 8us) server_negotiation.cc:407] Connection header received
0807 15:12:04.073175 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:12:04.073176 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:12:04.073184 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:12:04.073229 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:12:04.073540 (+ 311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:04.074004 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:04.074709 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:04.074965 (+ 256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:04.075034 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:12:04.077583 (+ 2549us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.194.3:41819: BlockingRecv error: recv got EOF from 127.29.194.3:41819 (error 108)
Metrics: {"server-negotiator.queue_time_us":35}
I20260807 15:12:04.078928 11022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:04.071715 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37006 (local address 127.29.194.62:34011)
0807 15:12:04.071789 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0807 15:12:04.071791 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:12:04.071797 (+ 6us) server_negotiation.cc:407] Connection header received
0807 15:12:04.071818 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:12:04.071819 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:12:04.071829 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:12:04.071885 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:12:04.072502 (+ 617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:04.073252 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:04.074044 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:04.074252 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:04.074306 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:12:04.075584 (+ 1278us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:12:04.075590 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:12:04.075590 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:12:04.075604 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:12:04.077386 (+ 1782us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:12:04.077827 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:12:04.077831 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:12:04.077831 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:12:04.077960 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:12:04.078165 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:12:04.078166 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:12:04.078166 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:12:04.078416 (+ 250us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:12:04.078596 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0807 15:12:04.078718 (+ 122us) server_negotiation.cc:299] Negotiation successful
0807 15:12:04.078832 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
Aug 07 15:12:04 dist-test-slave-42z9 krb5kdc[9940](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.194.3@KRBTEST.COM for oryx/127.29.194.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 15:12:04 dist-test-slave-42z9 krb5kdc[9940](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.194.3@KRBTEST.COM for oryx/127.29.194.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 15:12:04.082471 11412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:04.077855 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:34011 (local address 127.29.194.3:49037)
0807 15:12:04.078043 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0807 15:12:04.078052 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0807 15:12:04.078090 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:12:04.078199 (+ 109us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:12:04.078199 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:12:04.078323 (+ 124us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:12:04.078356 (+ 33us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:12:04.078357 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:12:04.078987 (+ 630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:12:04.078989 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:12:04.079604 (+ 615us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:12:04.079607 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:12:04.080004 (+ 397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:12:04.080005 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:12:04.080048 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:12:04.080433 (+ 385us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:12:04.080438 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:12:04.082414 (+ 1976us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.194.62:34011: Server oryx/127.29.194.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":172}
W20260807 15:12:04.082588 11020 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:12:04.077916 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.194.3:49037 (local address 127.29.194.62:34011)
0807 15:12:04.077977 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0807 15:12:04.077979 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:12:04.078087 (+ 108us) server_negotiation.cc:407] Connection header received
0807 15:12:04.078133 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:12:04.078134 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:12:04.078144 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:12:04.078189 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:12:04.078416 (+ 227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:04.078888 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:04.079676 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:04.079939 (+ 263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:04.079994 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:12:04.082533 (+ 2539us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.194.3:49037: BlockingRecv error: recv got EOF from 127.29.194.3:49037 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
W20260807 15:12:04.082623 11408 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 07 15:12:04 dist-test-slave-42z9 krb5kdc[9940](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.194.1@KRBTEST.COM for oryx/127.29.194.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 15:12:04 dist-test-slave-42z9 krb5kdc[9940](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.194.1@KRBTEST.COM for oryx/127.29.194.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 15:12:04.791138 11415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:04.784765 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:34011 (local address 127.29.194.1:57631)
0807 15:12:04.784963 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0807 15:12:04.784975 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 15:12:04.785042 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:12:04.785247 (+ 205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:12:04.785248 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:12:04.785513 (+ 265us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:12:04.785592 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:12:04.785594 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:12:04.786460 (+ 866us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:12:04.786461 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:12:04.787270 (+ 809us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:12:04.787278 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:12:04.787907 (+ 629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:12:04.787909 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:12:04.787969 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:12:04.788702 (+ 733us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:12:04.788708 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:12:04.790999 (+ 2291us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.194.62:34011: Server oryx/127.29.194.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":103,"threads_started":1}
W20260807 15:12:04.791246 11416 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:12:04.784931 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.194.1:57631 (local address 127.29.194.62:34011)
0807 15:12:04.785080 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0807 15:12:04.785082 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:12:04.785093 (+ 11us) server_negotiation.cc:407] Connection header received
0807 15:12:04.785122 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:12:04.785122 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:12:04.785134 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:12:04.785228 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:12:04.785687 (+ 459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:04.786293 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:04.787415 (+ 1122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:04.787774 (+ 359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:04.787868 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:12:04.791168 (+ 3300us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.194.1:57631: BlockingRecv error: recv got EOF from 127.29.194.1:57631 (error 108)
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":82,"threads_started":1}
Aug 07 15:12:04 dist-test-slave-42z9 krb5kdc[9940](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.194.2@KRBTEST.COM for oryx/127.29.194.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 15:12:04 dist-test-slave-42z9 krb5kdc[9940](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.194.2@KRBTEST.COM for oryx/127.29.194.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 15:12:04.927455 11417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:04.922625 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:34011 (local address 127.29.194.2:50555)
0807 15:12:04.922749 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0807 15:12:04.922759 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 15:12:04.922809 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:12:04.922964 (+ 155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:12:04.922964 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:12:04.923129 (+ 165us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:12:04.923180 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:12:04.923182 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:12:04.924104 (+ 922us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:12:04.924106 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:12:04.924702 (+ 596us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:12:04.924705 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:12:04.925040 (+ 335us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:12:04.925041 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:12:04.925096 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:12:04.925574 (+ 478us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:12:04.925580 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:12:04.927355 (+ 1775us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.194.62:34011: Server oryx/127.29.194.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":64,"threads_started":1}
W20260807 15:12:04.927522 11416 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:12:04.922728 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.194.2:50555 (local address 127.29.194.62:34011)
0807 15:12:04.922781 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0807 15:12:04.922783 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:12:04.922796 (+ 13us) server_negotiation.cc:407] Connection header received
0807 15:12:04.922866 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:12:04.922867 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:12:04.922878 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:12:04.922940 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:12:04.923434 (+ 494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:04.923991 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:04.924773 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:04.924972 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:04.925026 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:12:04.927465 (+ 2439us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.194.2:50555: BlockingRecv error: recv got EOF from 127.29.194.2:50555 (error 108)
Metrics: {"server-negotiator.queue_time_us":27}
Aug 07 15:12:05 dist-test-slave-42z9 krb5kdc[9940](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.194.3@KRBTEST.COM for oryx/127.29.194.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 15:12:05 dist-test-slave-42z9 krb5kdc[9940](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.194.3@KRBTEST.COM for oryx/127.29.194.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 15:12:05.088099 11418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:05.083070 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:34011 (local address 127.29.194.3:58159)
0807 15:12:05.083212 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0807 15:12:05.083227 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 15:12:05.083293 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:12:05.083470 (+ 177us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:12:05.083471 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:12:05.083643 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:12:05.083695 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:12:05.083696 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:12:05.084418 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:12:05.084419 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:12:05.084990 (+ 571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:12:05.084993 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:12:05.085601 (+ 608us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:12:05.085602 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:12:05.085659 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:12:05.086133 (+ 474us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:12:05.086138 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:12:05.088021 (+ 1883us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.194.62:34011: Server oryx/127.29.194.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":77,"threads_started":1}
W20260807 15:12:05.088281 11416 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:12:05.083187 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.194.3:58159 (local address 127.29.194.62:34011)
0807 15:12:05.083272 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0807 15:12:05.083274 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:12:05.083286 (+ 12us) server_negotiation.cc:407] Connection header received
0807 15:12:05.083346 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:12:05.083347 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:12:05.083358 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:12:05.083458 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:12:05.083800 (+ 342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:05.084305 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:05.085187 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:05.085435 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:05.085569 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:12:05.088221 (+ 2652us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.194.3:58159: BlockingRecv error: recv got EOF from 127.29.194.3:58159 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Aug 07 15:12:05 dist-test-slave-42z9 krb5kdc[9940](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.194.1@KRBTEST.COM for oryx/127.29.194.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 15:12:05 dist-test-slave-42z9 krb5kdc[9940](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.194.1@KRBTEST.COM for oryx/127.29.194.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 15:12:05.797212 11419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:05.791817 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:34011 (local address 127.29.194.1:39453)
0807 15:12:05.792033 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0807 15:12:05.792048 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 15:12:05.792136 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:12:05.792386 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:12:05.792388 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:12:05.792571 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:12:05.792632 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:12:05.792634 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:12:05.793419 (+ 785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:12:05.793420 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:12:05.794005 (+ 585us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:12:05.794008 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:12:05.794518 (+ 510us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:12:05.794519 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:12:05.794573 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:12:05.795167 (+ 594us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:12:05.795174 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:12:05.797148 (+ 1974us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.194.62:34011: Server oryx/127.29.194.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":80,"threads_started":1}
W20260807 15:12:05.797371 11420 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:12:05.791970 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.194.1:39453 (local address 127.29.194.62:34011)
0807 15:12:05.792104 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0807 15:12:05.792106 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:12:05.792119 (+ 13us) server_negotiation.cc:407] Connection header received
0807 15:12:05.792207 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:12:05.792208 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:12:05.792222 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:12:05.792293 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:12:05.792789 (+ 496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:05.793309 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:05.794143 (+ 834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:05.794415 (+ 272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:05.794502 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:12:05.797299 (+ 2797us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.194.1:39453: BlockingRecv error: recv got EOF from 127.29.194.1:39453 (error 108)
Metrics: {"server-negotiator.queue_time_us":110,"thread_start_us":64,"threads_started":1}
Aug 07 15:12:05 dist-test-slave-42z9 krb5kdc[9940](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.194.2@KRBTEST.COM for oryx/127.29.194.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 15:12:05 dist-test-slave-42z9 krb5kdc[9940](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.194.2@KRBTEST.COM for oryx/127.29.194.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 15:12:05.933204 11421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:05.928181 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:34011 (local address 127.29.194.2:60185)
0807 15:12:05.928339 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0807 15:12:05.928352 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 15:12:05.928440 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:12:05.928584 (+ 144us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:12:05.928584 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:12:05.928828 (+ 244us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:12:05.928897 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:12:05.928900 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:12:05.929658 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:12:05.929659 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:12:05.930363 (+ 704us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:12:05.930373 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:12:05.930845 (+ 472us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:12:05.930847 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:12:05.930909 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:12:05.931396 (+ 487us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:12:05.931401 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:12:05.933120 (+ 1719us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.194.62:34011: Server oryx/127.29.194.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":96,"threads_started":1}
W20260807 15:12:05.933391 11420 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:12:05.928251 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.194.2:60185 (local address 127.29.194.62:34011)
0807 15:12:05.928311 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0807 15:12:05.928313 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:12:05.928418 (+ 105us) server_negotiation.cc:407] Connection header received
0807 15:12:05.928507 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:12:05.928508 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:12:05.928522 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:12:05.928570 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:12:05.928986 (+ 416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:05.929574 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:05.930552 (+ 978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:05.930770 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:05.930835 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:12:05.933338 (+ 2503us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.194.2:60185: BlockingRecv error: recv got EOF from 127.29.194.2:60185 (error 108)
Metrics: {"server-negotiator.queue_time_us":26}
Aug 07 15:12:06 dist-test-slave-42z9 krb5kdc[9940](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.194.3@KRBTEST.COM for oryx/127.29.194.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 15:12:06 dist-test-slave-42z9 krb5kdc[9940](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.194.3@KRBTEST.COM for oryx/127.29.194.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 15:12:06.093825 11423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:06.088702 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:34011 (local address 127.29.194.3:48817)
0807 15:12:06.088906 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0807 15:12:06.088921 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 15:12:06.088995 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:12:06.089150 (+ 155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:12:06.089151 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:12:06.089332 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:12:06.089386 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:12:06.089387 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:12:06.090122 (+ 735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:12:06.090124 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:12:06.090676 (+ 552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:12:06.090679 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:12:06.091141 (+ 462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:12:06.091143 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:12:06.091233 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:12:06.091839 (+ 606us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:12:06.091845 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:12:06.093759 (+ 1914us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.194.62:34011: Server oryx/127.29.194.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":70,"threads_started":1}
W20260807 15:12:06.093931 11420 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:12:06.088941 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.194.3:48817 (local address 127.29.194.62:34011)
0807 15:12:06.088989 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0807 15:12:06.088992 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:12:06.089004 (+ 12us) server_negotiation.cc:407] Connection header received
0807 15:12:06.089055 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:12:06.089056 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:12:06.089073 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:12:06.089131 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:12:06.089472 (+ 341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:06.090020 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:06.090796 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:06.090991 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:06.091052 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:12:06.093869 (+ 2817us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.194.3:48817: BlockingRecv error: recv got EOF from 127.29.194.3:48817 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
Aug 07 15:12:06 dist-test-slave-42z9 krb5kdc[9940](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.194.1@KRBTEST.COM for oryx/127.29.194.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 15:12:06 dist-test-slave-42z9 krb5kdc[9940](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.194.1@KRBTEST.COM for oryx/127.29.194.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 15:12:06.807072 11424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:06.797848 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:34011 (local address 127.29.194.1:54917)
0807 15:12:06.797988 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0807 15:12:06.797998 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 15:12:06.798052 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:12:06.798265 (+ 213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:12:06.798266 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:12:06.798442 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:12:06.798494 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:12:06.798496 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:12:06.799199 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:12:06.799200 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:12:06.799788 (+ 588us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:12:06.799790 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:12:06.800487 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:12:06.800489 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:12:06.800561 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:12:06.804587 (+ 4026us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:12:06.804594 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:12:06.806986 (+ 2392us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.194.62:34011: Server oryx/127.29.194.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":76,"threads_started":1}
W20260807 15:12:06.807224 11425 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:12:06.797954 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.194.1:54917 (local address 127.29.194.62:34011)
0807 15:12:06.798125 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0807 15:12:06.798127 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:12:06.798137 (+ 10us) server_negotiation.cc:407] Connection header received
0807 15:12:06.798164 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:12:06.798165 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:12:06.798176 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:12:06.798247 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:12:06.798592 (+ 345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:06.799109 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:06.799953 (+ 844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:06.800359 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:06.800427 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:12:06.807160 (+ 6733us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.194.1:54917: BlockingRecv error: recv got EOF from 127.29.194.1:54917 (error 108)
Metrics: {"server-negotiator.queue_time_us":151,"thread_start_us":80,"threads_started":1}
Aug 07 15:12:06 dist-test-slave-42z9 krb5kdc[9940](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.194.2@KRBTEST.COM for oryx/127.29.194.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 15:12:06 dist-test-slave-42z9 krb5kdc[9940](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.194.2@KRBTEST.COM for oryx/127.29.194.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 15:12:06.938908 11427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:06.933915 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:34011 (local address 127.29.194.2:43277)
0807 15:12:06.934073 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0807 15:12:06.934087 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 15:12:06.934167 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:12:06.934297 (+ 130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:12:06.934298 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:12:06.934479 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:12:06.934537 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:12:06.934539 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:12:06.935333 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:12:06.935335 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:12:06.935995 (+ 660us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:12:06.935998 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:12:06.936351 (+ 353us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:12:06.936352 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:12:06.936407 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:12:06.936868 (+ 461us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:12:06.936872 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:12:06.938828 (+ 1956us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.194.62:34011: Server oryx/127.29.194.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":71,"threads_started":1}
W20260807 15:12:06.939000 11425 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:12:06.934030 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.194.2:43277 (local address 127.29.194.62:34011)
0807 15:12:06.934092 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0807 15:12:06.934094 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:12:06.934147 (+ 53us) server_negotiation.cc:407] Connection header received
0807 15:12:06.934217 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:12:06.934218 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:12:06.934230 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:12:06.934278 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:12:06.934604 (+ 326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:06.935263 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:06.936083 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:06.936284 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:06.936340 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:12:06.938945 (+ 2605us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.194.2:43277: BlockingRecv error: recv got EOF from 127.29.194.2:43277 (error 108)
Metrics: {"server-negotiator.queue_time_us":32}
Aug 07 15:12:07 dist-test-slave-42z9 krb5kdc[9940](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.194.3@KRBTEST.COM for oryx/127.29.194.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 15:12:07 dist-test-slave-42z9 krb5kdc[9940](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.194.3@KRBTEST.COM for oryx/127.29.194.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 15:12:07.100425 11428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:07.094551 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:34011 (local address 127.29.194.3:55629)
0807 15:12:07.094780 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0807 15:12:07.094794 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 15:12:07.094891 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:12:07.095080 (+ 189us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:12:07.095081 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:12:07.095300 (+ 219us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:12:07.095376 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:12:07.095377 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:12:07.096151 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:12:07.096153 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:12:07.096731 (+ 578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:12:07.096733 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:12:07.097173 (+ 440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:12:07.097174 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:12:07.097222 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:12:07.097807 (+ 585us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:12:07.097813 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:12:07.100274 (+ 2461us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.194.62:34011: Server oryx/127.29.194.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":136,"threads_started":1}
W20260807 15:12:07.100519 11425 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:12:07.094596 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.194.3:55629 (local address 127.29.194.62:34011)
0807 15:12:07.094662 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0807 15:12:07.094664 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:12:07.094882 (+ 218us) server_negotiation.cc:407] Connection header received
0807 15:12:07.094974 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:12:07.094975 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:12:07.094992 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:12:07.095056 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:12:07.095496 (+ 440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:07.096016 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:07.096876 (+ 860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:07.097081 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:07.097146 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:12:07.100450 (+ 3304us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.194.3:55629: BlockingRecv error: recv got EOF from 127.29.194.3:55629 (error 108)
Metrics: {"server-negotiator.queue_time_us":34}
Aug 07 15:12:07 dist-test-slave-42z9 krb5kdc[9940](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.194.1@KRBTEST.COM for oryx/127.29.194.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 15:12:07 dist-test-slave-42z9 krb5kdc[9940](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.194.1@KRBTEST.COM for oryx/127.29.194.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 15:12:07.813326 11429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:07.807946 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:34011 (local address 127.29.194.1:43591)
0807 15:12:07.808133 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0807 15:12:07.808224 (+ 91us) client_negotiation.cc:174] Beginning negotiation
0807 15:12:07.808341 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:12:07.808571 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:12:07.808572 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:12:07.808755 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:12:07.808811 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:12:07.808813 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:12:07.809500 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:12:07.809501 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:12:07.810134 (+ 633us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:12:07.810138 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:12:07.810572 (+ 434us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:12:07.810573 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:12:07.810641 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:12:07.811133 (+ 492us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:12:07.811138 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:12:07.813262 (+ 2124us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.194.62:34011: Server oryx/127.29.194.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":105,"threads_started":1}
W20260807 15:12:07.813527 11430 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:12:07.808114 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.194.1:43591 (local address 127.29.194.62:34011)
0807 15:12:07.808368 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0807 15:12:07.808370 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:12:07.808382 (+ 12us) server_negotiation.cc:407] Connection header received
0807 15:12:07.808411 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:12:07.808412 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:12:07.808424 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:12:07.808504 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:12:07.808879 (+ 375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:07.809398 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:07.810220 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:07.810471 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:07.810536 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:12:07.813409 (+ 2873us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.194.1:43591: BlockingRecv error: recv got EOF from 127.29.194.1:43591 (error 108)
Metrics: {"server-negotiator.queue_time_us":215,"thread_start_us":88,"threads_started":1}
Aug 07 15:12:07 dist-test-slave-42z9 krb5kdc[9940](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.194.2@KRBTEST.COM for oryx/127.29.194.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 15:12:07 dist-test-slave-42z9 krb5kdc[9940](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.194.2@KRBTEST.COM for oryx/127.29.194.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 15:12:07.945063 11432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:07.939486 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:34011 (local address 127.29.194.2:57543)
0807 15:12:07.939742 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0807 15:12:07.939759 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 15:12:07.939917 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:12:07.940085 (+ 168us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:12:07.940087 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:12:07.940336 (+ 249us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:12:07.940406 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:12:07.940408 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:12:07.941205 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:12:07.941206 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:12:07.941947 (+ 741us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:12:07.941955 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:12:07.942419 (+ 464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:12:07.942420 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:12:07.942478 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:12:07.942945 (+ 467us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:12:07.942950 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:12:07.944998 (+ 2048us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.194.62:34011: Server oryx/127.29.194.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":225,"thread_start_us":68,"threads_started":1}
W20260807 15:12:07.945312 11430 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:12:07.939639 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.194.2:57543 (local address 127.29.194.62:34011)
0807 15:12:07.939701 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0807 15:12:07.939703 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:12:07.939900 (+ 197us) server_negotiation.cc:407] Connection header received
0807 15:12:07.939999 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:12:07.940000 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:12:07.940018 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:12:07.940072 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:12:07.940488 (+ 416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:07.941112 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:07.942101 (+ 989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:07.942323 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:07.942390 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:12:07.945221 (+ 2831us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.194.2:57543: BlockingRecv error: recv got EOF from 127.29.194.2:57543 (error 108)
Metrics: {"server-negotiator.queue_time_us":35}
Aug 07 15:12:08 dist-test-slave-42z9 krb5kdc[9940](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.194.3@KRBTEST.COM for oryx/127.29.194.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 15:12:08 dist-test-slave-42z9 krb5kdc[9940](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.194.3@KRBTEST.COM for oryx/127.29.194.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 15:12:08.106086 11433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:08.101023 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:34011 (local address 127.29.194.3:42977)
0807 15:12:08.101156 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0807 15:12:08.101167 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 15:12:08.101234 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:12:08.101380 (+ 146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:12:08.101381 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:12:08.101607 (+ 226us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:12:08.101662 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:12:08.101664 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:12:08.102324 (+ 660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:12:08.102325 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:12:08.102849 (+ 524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:12:08.102851 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:12:08.103181 (+ 330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:12:08.103182 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:12:08.103234 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:12:08.103706 (+ 472us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:12:08.103711 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:12:08.106004 (+ 2293us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.194.62:34011: Server oryx/127.29.194.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":74,"threads_started":1}
W20260807 15:12:08.106254 11430 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:12:08.101062 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.194.3:42977 (local address 127.29.194.62:34011)
0807 15:12:08.101130 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0807 15:12:08.101132 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:12:08.101223 (+ 91us) server_negotiation.cc:407] Connection header received
0807 15:12:08.101311 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:12:08.101311 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:12:08.101325 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:12:08.101373 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:12:08.101732 (+ 359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:08.102248 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:08.102928 (+ 680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:08.103120 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:08.103175 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:12:08.106166 (+ 2991us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.194.3:42977: BlockingRecv error: recv got EOF from 127.29.194.3:42977 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
Aug 07 15:12:08 dist-test-slave-42z9 krb5kdc[9940](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.194.1@KRBTEST.COM for oryx/127.29.194.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 15:12:08 dist-test-slave-42z9 krb5kdc[9940](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.194.1@KRBTEST.COM for oryx/127.29.194.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 15:12:08.818895 11435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:08.813993 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:34011 (local address 127.29.194.1:34159)
0807 15:12:08.814153 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0807 15:12:08.814167 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 15:12:08.814243 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:12:08.814385 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:12:08.814386 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:12:08.814582 (+ 196us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:12:08.814636 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:12:08.814638 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:12:08.815303 (+ 665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:12:08.815304 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:12:08.815861 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:12:08.815863 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:12:08.816308 (+ 445us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:12:08.816309 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:12:08.816370 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:12:08.816895 (+ 525us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:12:08.816900 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:12:08.818830 (+ 1930us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.194.62:34011: Server oryx/127.29.194.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":70,"threads_started":1}
W20260807 15:12:08.819033 11436 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:12:08.814110 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.194.1:34159 (local address 127.29.194.62:34011)
0807 15:12:08.814238 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0807 15:12:08.814239 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 15:12:08.814250 (+ 11us) server_negotiation.cc:407] Connection header received
0807 15:12:08.814302 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:12:08.814303 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:12:08.814314 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:12:08.814373 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:12:08.814700 (+ 327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:08.815223 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:08.815981 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:08.816236 (+ 255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:08.816296 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:12:08.818947 (+ 2651us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.194.1:34159: BlockingRecv error: recv got EOF from 127.29.194.1:34159 (error 108)
Metrics: {"server-negotiator.queue_time_us":102,"thread_start_us":71,"threads_started":1}
Aug 07 15:12:08 dist-test-slave-42z9 krb5kdc[9940](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.194.2@KRBTEST.COM for oryx/127.29.194.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 15:12:08 dist-test-slave-42z9 krb5kdc[9940](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.194.2@KRBTEST.COM for oryx/127.29.194.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 15:12:08.950838 11437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:08.945815 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:34011 (local address 127.29.194.2:56785)
0807 15:12:08.945984 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0807 15:12:08.945998 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 15:12:08.946075 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:12:08.946401 (+ 326us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:12:08.946403 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:12:08.946616 (+ 213us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:12:08.946672 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:12:08.946674 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:12:08.947346 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:12:08.947347 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:12:08.947894 (+ 547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:12:08.947896 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:12:08.948252 (+ 356us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:12:08.948253 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:12:08.948308 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:12:08.948773 (+ 465us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:12:08.948777 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:12:08.950767 (+ 1990us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.194.62:34011: Server oryx/127.29.194.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":84,"threads_started":1}
W20260807 15:12:08.951191 11436 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:12:08.945970 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.194.2:56785 (local address 127.29.194.62:34011)
0807 15:12:08.946048 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0807 15:12:08.946050 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:12:08.946063 (+ 13us) server_negotiation.cc:407] Connection header received
0807 15:12:08.946256 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:12:08.946259 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:12:08.946287 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:12:08.946374 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:12:08.946759 (+ 385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:08.947264 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:08.947979 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:08.948186 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:08.948240 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:12:08.951105 (+ 2865us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.194.2:56785: BlockingRecv error: recv got EOF from 127.29.194.2:56785 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
Aug 07 15:12:09 dist-test-slave-42z9 krb5kdc[9940](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.194.3@KRBTEST.COM for oryx/127.29.194.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 15:12:09 dist-test-slave-42z9 krb5kdc[9940](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.194.3@KRBTEST.COM for oryx/127.29.194.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 15:12:09.113106 11438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:09.106773 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:34011 (local address 127.29.194.3:44547)
0807 15:12:09.106987 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0807 15:12:09.107006 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 15:12:09.107130 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:12:09.107333 (+ 203us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:12:09.107335 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:12:09.107574 (+ 239us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:12:09.107658 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:12:09.107660 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:12:09.108781 (+ 1121us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:12:09.108783 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:12:09.109432 (+ 649us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:12:09.109473 (+ 41us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:12:09.109961 (+ 488us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:12:09.109963 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:12:09.110031 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:12:09.110634 (+ 603us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:12:09.110638 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:12:09.113029 (+ 2391us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.194.62:34011: Server oryx/127.29.194.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":83,"threads_started":1}
W20260807 15:12:09.113268 11436 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:12:09.106937 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.194.3:44547 (local address 127.29.194.62:34011)
0807 15:12:09.107027 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0807 15:12:09.107029 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:12:09.107107 (+ 78us) server_negotiation.cc:407] Connection header received
0807 15:12:09.107196 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:12:09.107198 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:12:09.107212 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:12:09.107276 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:12:09.107769 (+ 493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:09.108585 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:09.109594 (+ 1009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:09.109829 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:09.109911 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:12:09.113191 (+ 3280us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.194.3:44547: BlockingRecv error: recv got EOF from 127.29.194.3:44547 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Aug 07 15:12:09 dist-test-slave-42z9 krb5kdc[9940](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.194.1@KRBTEST.COM for oryx/127.29.194.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 15:12:09 dist-test-slave-42z9 krb5kdc[9940](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.194.1@KRBTEST.COM for oryx/127.29.194.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 15:12:09.825230 11439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:09.819637 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:34011 (local address 127.29.194.1:46551)
0807 15:12:09.819831 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0807 15:12:09.819845 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 15:12:09.819929 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:12:09.820240 (+ 311us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:12:09.820241 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:12:09.820427 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:12:09.820486 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:12:09.820487 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:12:09.821375 (+ 888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:12:09.821376 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:12:09.822023 (+ 647us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:12:09.822026 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:12:09.822493 (+ 467us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:12:09.822495 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:12:09.822581 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:12:09.823260 (+ 679us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:12:09.823266 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:12:09.825166 (+ 1900us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.194.62:34011: Server oryx/127.29.194.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":113,"threads_started":1}
W20260807 15:12:09.825409 11440 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:12:09.819800 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.194.1:46551 (local address 127.29.194.62:34011)
0807 15:12:09.819964 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0807 15:12:09.819967 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:12:09.819984 (+ 17us) server_negotiation.cc:407] Connection header received
0807 15:12:09.820024 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:12:09.820025 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:12:09.820042 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:12:09.820206 (+ 164us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:12:09.820630 (+ 424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:09.821250 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:09.822135 (+ 885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:09.822347 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:09.822414 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:12:09.825326 (+ 2912us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.194.1:46551: BlockingRecv error: recv got EOF from 127.29.194.1:46551 (error 108)
Metrics: {"server-negotiator.queue_time_us":131,"thread_start_us":73,"threads_started":1}
Aug 07 15:12:09 dist-test-slave-42z9 krb5kdc[9940](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.194.2@KRBTEST.COM for oryx/127.29.194.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 15:12:09 dist-test-slave-42z9 krb5kdc[9940](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.194.2@KRBTEST.COM for oryx/127.29.194.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 15:12:09.956693 11441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:09.951486 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:34011 (local address 127.29.194.2:40105)
0807 15:12:09.951668 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0807 15:12:09.951678 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 15:12:09.951739 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:12:09.951883 (+ 144us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:12:09.951884 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:12:09.952055 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:12:09.952106 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:12:09.952107 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:12:09.953118 (+ 1011us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:12:09.953119 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:12:09.953720 (+ 601us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:12:09.953723 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:12:09.954117 (+ 394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:12:09.954117 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:12:09.954158 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:12:09.954671 (+ 513us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:12:09.954676 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:12:09.956630 (+ 1954us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.194.62:34011: Server oryx/127.29.194.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":92,"threads_started":1}
W20260807 15:12:09.956800 11440 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:12:09.951590 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.194.2:40105 (local address 127.29.194.62:34011)
0807 15:12:09.951654 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0807 15:12:09.951656 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:12:09.951737 (+ 81us) server_negotiation.cc:407] Connection header received
0807 15:12:09.951792 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:12:09.951793 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:12:09.951805 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:12:09.951867 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:12:09.952252 (+ 385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:09.952991 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:09.953819 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:09.954053 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:09.954111 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:12:09.956739 (+ 2628us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.194.2:40105: BlockingRecv error: recv got EOF from 127.29.194.2:40105 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
Aug 07 15:12:10 dist-test-slave-42z9 krb5kdc[9940](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.194.3@KRBTEST.COM for oryx/127.29.194.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 15:12:10 dist-test-slave-42z9 krb5kdc[9940](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.194.3@KRBTEST.COM for oryx/127.29.194.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 15:12:10.119002 11442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:10.113804 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:34011 (local address 127.29.194.3:53567)
0807 15:12:10.114018 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0807 15:12:10.114031 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 15:12:10.114119 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:12:10.114260 (+ 141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:12:10.114261 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:12:10.114489 (+ 228us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:12:10.114542 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:12:10.114544 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:12:10.115226 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:12:10.115227 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:12:10.115753 (+ 526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:12:10.115755 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:12:10.116107 (+ 352us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:12:10.116108 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:12:10.116161 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:12:10.116703 (+ 542us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:12:10.116708 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:12:10.118924 (+ 2216us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.194.62:34011: Server oryx/127.29.194.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":106,"threads_started":1}
W20260807 15:12:10.119092 11440 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:12:10.113963 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.194.3:53567 (local address 127.29.194.62:34011)
0807 15:12:10.114030 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0807 15:12:10.114032 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:12:10.114096 (+ 64us) server_negotiation.cc:407] Connection header received
0807 15:12:10.114177 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:12:10.114179 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:12:10.114193 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:12:10.114243 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:12:10.114624 (+ 381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:10.115142 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:10.115863 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:10.116042 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:10.116099 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:12:10.119034 (+ 2935us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.194.3:53567: BlockingRecv error: recv got EOF from 127.29.194.3:53567 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
Aug 07 15:12:10 dist-test-slave-42z9 krb5kdc[9940](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.194.1@KRBTEST.COM for oryx/127.29.194.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 15:12:10 dist-test-slave-42z9 krb5kdc[9940](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.194.1@KRBTEST.COM for oryx/127.29.194.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 15:12:10.832103 11443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:10.825920 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:34011 (local address 127.29.194.1:59771)
0807 15:12:10.826139 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0807 15:12:10.826151 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 15:12:10.826217 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:12:10.826480 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:12:10.826482 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:12:10.826680 (+ 198us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:12:10.826749 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:12:10.826751 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:12:10.827882 (+ 1131us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:12:10.827883 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:12:10.828704 (+ 821us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:12:10.828708 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:12:10.829279 (+ 571us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:12:10.829281 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:12:10.829395 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:12:10.829907 (+ 512us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:12:10.829912 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:12:10.832029 (+ 2117us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.194.62:34011: Server oryx/127.29.194.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":100,"threads_started":1}
W20260807 15:12:10.832394 11445 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:12:10.826067 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.194.1:59771 (local address 127.29.194.62:34011)
0807 15:12:10.826219 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0807 15:12:10.826221 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:12:10.826268 (+ 47us) server_negotiation.cc:407] Connection header received
0807 15:12:10.826301 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:12:10.826302 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:12:10.826316 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:12:10.826378 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:12:10.827224 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:10.827755 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:10.828842 (+ 1087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:10.829155 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:10.829229 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:12:10.832294 (+ 3065us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.194.1:59771: BlockingRecv error: recv got EOF from 127.29.194.1:59771 (error 108)
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":93,"threads_started":1}
Aug 07 15:12:10 dist-test-slave-42z9 krb5kdc[9940](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.194.2@KRBTEST.COM for oryx/127.29.194.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 15:12:10 dist-test-slave-42z9 krb5kdc[9940](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.194.2@KRBTEST.COM for oryx/127.29.194.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 15:12:10.963328 11449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:10.957429 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:34011 (local address 127.29.194.2:56463)
0807 15:12:10.957695 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0807 15:12:10.957708 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 15:12:10.957790 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:12:10.958033 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:12:10.958034 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:12:10.958282 (+ 248us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:12:10.958368 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:12:10.958370 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:12:10.959295 (+ 925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:12:10.959297 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:12:10.959938 (+ 641us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:12:10.959941 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:12:10.960416 (+ 475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:12:10.960418 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:12:10.960470 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:12:10.961056 (+ 586us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:12:10.961063 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:12:10.963232 (+ 2169us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.194.62:34011: Server oryx/127.29.194.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":241,"thread_start_us":101,"threads_started":1}
W20260807 15:12:10.963466 11445 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:12:10.957595 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.194.2:56463 (local address 127.29.194.62:34011)
0807 15:12:10.957659 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0807 15:12:10.957662 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:12:10.957789 (+ 127us) server_negotiation.cc:407] Connection header received
0807 15:12:10.957850 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:12:10.957850 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:12:10.957864 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:12:10.957948 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:12:10.958506 (+ 558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:10.959134 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:10.960086 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:10.960320 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:10.960393 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:12:10.963396 (+ 3003us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.194.2:56463: BlockingRecv error: recv got EOF from 127.29.194.2:56463 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
Aug 07 15:12:11 dist-test-slave-42z9 krb5kdc[9940](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.194.3@KRBTEST.COM for oryx/127.29.194.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 15:12:11 dist-test-slave-42z9 krb5kdc[9940](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.194.3@KRBTEST.COM for oryx/127.29.194.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 15:12:11.125814 11450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:11.119721 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:34011 (local address 127.29.194.3:54961)
0807 15:12:11.119877 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0807 15:12:11.119891 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 15:12:11.119963 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:12:11.120093 (+ 130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:12:11.120094 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:12:11.120266 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:12:11.120320 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:12:11.120322 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:12:11.121065 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:12:11.121067 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:12:11.121818 (+ 751us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:12:11.121822 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:12:11.122412 (+ 590us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:12:11.122413 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:12:11.122468 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:12:11.122983 (+ 515us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:12:11.122988 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:12:11.125711 (+ 2723us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.194.62:34011: Server oryx/127.29.194.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":82,"threads_started":1}
W20260807 15:12:11.125905 11445 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:12:11.119855 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.194.3:54961 (local address 127.29.194.62:34011)
0807 15:12:11.119917 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0807 15:12:11.119919 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:12:11.119946 (+ 27us) server_negotiation.cc:407] Connection header received
0807 15:12:11.120022 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:12:11.120023 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:12:11.120036 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:12:11.120076 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:12:11.120451 (+ 375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:11.120966 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:11.122018 (+ 1052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:11.122277 (+ 259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:11.122384 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:12:11.125838 (+ 3454us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.194.3:54961: BlockingRecv error: recv got EOF from 127.29.194.3:54961 (error 108)
Metrics: {"server-negotiator.queue_time_us":32}
Aug 07 15:12:11 dist-test-slave-42z9 krb5kdc[9940](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.194.1@KRBTEST.COM for oryx/127.29.194.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 15:12:11 dist-test-slave-42z9 krb5kdc[9940](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.194.1@KRBTEST.COM for oryx/127.29.194.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 15:12:11.841828 11451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:11.832683 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:34011 (local address 127.29.194.1:43655)
0807 15:12:11.832817 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0807 15:12:11.832829 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 15:12:11.832886 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:12:11.833056 (+ 170us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:12:11.833057 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:12:11.833231 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:12:11.833287 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:12:11.833289 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:12:11.834370 (+ 1081us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:12:11.834371 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:12:11.834970 (+ 599us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:12:11.834972 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:12:11.835419 (+ 447us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:12:11.835421 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:12:11.835472 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:12:11.839695 (+ 4223us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:12:11.839703 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:12:11.841732 (+ 2029us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.194.62:34011: Server oryx/127.29.194.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":64,"threads_started":1}
W20260807 15:12:11.841954 11452 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:12:11.832819 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.194.1:43655 (local address 127.29.194.62:34011)
0807 15:12:11.832941 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0807 15:12:11.832943 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:12:11.832950 (+ 7us) server_negotiation.cc:407] Connection header received
0807 15:12:11.832972 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:12:11.832972 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:12:11.832984 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:12:11.833046 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:12:11.833473 (+ 427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:11.834245 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:11.835097 (+ 852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:11.835330 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:11.835403 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:12:11.841888 (+ 6485us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.194.1:43655: BlockingRecv error: recv got EOF from 127.29.194.1:43655 (error 108)
Metrics: {"server-negotiator.queue_time_us":102,"thread_start_us":69,"threads_started":1}
Aug 07 15:12:11 dist-test-slave-42z9 krb5kdc[9940](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.194.2@KRBTEST.COM for oryx/127.29.194.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 15:12:11 dist-test-slave-42z9 krb5kdc[9940](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.194.2@KRBTEST.COM for oryx/127.29.194.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 15:12:11.970480 11454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:11.964338 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:34011 (local address 127.29.194.2:38805)
0807 15:12:11.964555 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0807 15:12:11.964575 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 15:12:11.964705 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:12:11.964896 (+ 191us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:12:11.964897 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:12:11.965154 (+ 257us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:12:11.965238 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:12:11.965241 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:12:11.966372 (+ 1131us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:12:11.966373 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:12:11.967069 (+ 696us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:12:11.967073 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:12:11.967700 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:12:11.967702 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:12:11.967791 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:12:11.968304 (+ 513us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:12:11.968310 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:12:11.970378 (+ 2068us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.194.62:34011: Server oryx/127.29.194.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":111,"threads_started":1}
W20260807 15:12:11.970578 11452 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:12:11.964517 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.194.2:38805 (local address 127.29.194.62:34011)
0807 15:12:11.964627 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0807 15:12:11.964630 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:12:11.964681 (+ 51us) server_negotiation.cc:407] Connection header received
0807 15:12:11.964804 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:12:11.964805 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:12:11.964823 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:12:11.964876 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:12:11.965387 (+ 511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:11.966288 (+ 901us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:11.967242 (+ 954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:11.967530 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:11.967656 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:12:11.970522 (+ 2866us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.194.2:38805: BlockingRecv error: recv got EOF from 127.29.194.2:38805 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
Aug 07 15:12:12 dist-test-slave-42z9 krb5kdc[9940](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.194.3@KRBTEST.COM for oryx/127.29.194.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 15:12:12 dist-test-slave-42z9 krb5kdc[9940](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.194.3@KRBTEST.COM for oryx/127.29.194.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 15:12:12.131628 11457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:12.126499 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:34011 (local address 127.29.194.3:33475)
0807 15:12:12.126758 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0807 15:12:12.126772 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 15:12:12.126850 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:12:12.127013 (+ 163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:12:12.127014 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:12:12.127190 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:12:12.127247 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:12:12.127249 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:12:12.127984 (+ 735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:12:12.127985 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:12:12.128528 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:12:12.128530 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:12:12.128900 (+ 370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:12:12.128901 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:12:12.128956 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:12:12.129507 (+ 551us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:12:12.129513 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:12:12.131539 (+ 2026us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.194.62:34011: Server oryx/127.29.194.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":235,"thread_start_us":197,"threads_started":1}
W20260807 15:12:12.131734 11452 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:12:12.126576 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.194.3:33475 (local address 127.29.194.62:34011)
0807 15:12:12.126644 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0807 15:12:12.126647 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:12:12.126890 (+ 243us) server_negotiation.cc:407] Connection header received
0807 15:12:12.126924 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:12:12.126925 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:12:12.126941 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:12:12.127002 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:12:12.127351 (+ 349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:12.127863 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:12.128620 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:12.128805 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:12.128867 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:12:12.131671 (+ 2804us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.194.3:33475: BlockingRecv error: recv got EOF from 127.29.194.3:33475 (error 108)
Metrics: {"server-negotiator.queue_time_us":30}
Aug 07 15:12:12 dist-test-slave-42z9 krb5kdc[9940](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.194.1@KRBTEST.COM for oryx/127.29.194.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 15:12:12 dist-test-slave-42z9 krb5kdc[9940](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.194.1@KRBTEST.COM for oryx/127.29.194.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 15:12:12.848340 11458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:12.842557 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:34011 (local address 127.29.194.1:56259)
0807 15:12:12.842789 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0807 15:12:12.842807 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 15:12:12.842890 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:12:12.843121 (+ 231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:12:12.843122 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:12:12.843304 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:12:12.843363 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:12:12.843364 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:12:12.844183 (+ 819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:12:12.844184 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:12:12.844765 (+ 581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:12:12.844767 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:12:12.845219 (+ 452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:12:12.845219 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:12:12.845266 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:12:12.845798 (+ 532us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:12:12.845805 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:12:12.848258 (+ 2453us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.194.62:34011: Server oryx/127.29.194.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":131,"threads_started":1}
W20260807 15:12:12.848448 11459 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:12:12.842686 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.194.1:56259 (local address 127.29.194.62:34011)
0807 15:12:12.842943 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0807 15:12:12.842946 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:12:12.842961 (+ 15us) server_negotiation.cc:407] Connection header received
0807 15:12:12.842996 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:12:12.842997 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:12:12.843013 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:12:12.843102 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:12:12.843446 (+ 344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:12.844067 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:12.844899 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:12.845137 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:12.845214 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:12:12.848381 (+ 3167us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.194.1:56259: BlockingRecv error: recv got EOF from 127.29.194.1:56259 (error 108)
Metrics: {"server-negotiator.queue_time_us":210,"thread_start_us":121,"threads_started":1}
Aug 07 15:12:12 dist-test-slave-42z9 krb5kdc[9940](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.194.2@KRBTEST.COM for oryx/127.29.194.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 15:12:12 dist-test-slave-42z9 krb5kdc[9940](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.194.2@KRBTEST.COM for oryx/127.29.194.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 15:12:12.975769 11460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:12.971062 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:34011 (local address 127.29.194.2:56997)
0807 15:12:12.971198 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0807 15:12:12.971210 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 15:12:12.971284 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:12:12.971443 (+ 159us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:12:12.971444 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:12:12.971625 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:12:12.971681 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:12:12.971683 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:12:12.972358 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:12:12.972359 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:12:12.972887 (+ 528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:12:12.972888 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:12:12.973213 (+ 325us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:12:12.973213 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:12:12.973266 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:12:12.973738 (+ 472us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:12:12.973742 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:12:12.975686 (+ 1944us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.194.62:34011: Server oryx/127.29.194.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":73,"threads_started":1}
W20260807 15:12:12.975853 11459 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:12:12.971107 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.194.2:56997 (local address 127.29.194.62:34011)
0807 15:12:12.971158 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0807 15:12:12.971160 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:12:12.971273 (+ 113us) server_negotiation.cc:407] Connection header received
0807 15:12:12.971340 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:12:12.971341 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:12:12.971355 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:12:12.971404 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:12:12.971761 (+ 357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:12.972273 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:12.972968 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:12.973151 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:12.973206 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:12:12.975797 (+ 2591us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.194.2:56997: BlockingRecv error: recv got EOF from 127.29.194.2:56997 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
Aug 07 15:12:13 dist-test-slave-42z9 krb5kdc[9940](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.194.3@KRBTEST.COM for oryx/127.29.194.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 15:12:13 dist-test-slave-42z9 krb5kdc[9940](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.194.3@KRBTEST.COM for oryx/127.29.194.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 15:12:13.137847 11461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:13.132243 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:34011 (local address 127.29.194.3:55423)
0807 15:12:13.132434 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0807 15:12:13.132448 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 15:12:13.132539 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:12:13.132684 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:12:13.132685 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:12:13.132916 (+ 231us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:12:13.132992 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:12:13.132994 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:12:13.133817 (+ 823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:12:13.133819 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:12:13.134489 (+ 670us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:12:13.134492 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:12:13.134992 (+ 500us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:12:13.134994 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:12:13.135069 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:12:13.135509 (+ 440us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:12:13.135514 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:12:13.137703 (+ 2189us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.194.62:34011: Server oryx/127.29.194.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":100,"threads_started":1}
W20260807 15:12:13.137873 11459 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:12:13.132284 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.194.3:55423 (local address 127.29.194.62:34011)
0807 15:12:13.132375 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0807 15:12:13.132378 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:12:13.132535 (+ 157us) server_negotiation.cc:407] Connection header received
0807 15:12:13.132589 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:12:13.132590 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:12:13.132605 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:12:13.132654 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:12:13.133101 (+ 447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:13.133688 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:13.134631 (+ 943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:13.134878 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:13.134947 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:12:13.137817 (+ 2870us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.194.3:55423: BlockingRecv error: recv got EOF from 127.29.194.3:55423 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
Aug 07 15:12:13 dist-test-slave-42z9 krb5kdc[9940](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.194.1@KRBTEST.COM for oryx/127.29.194.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 15:12:13 dist-test-slave-42z9 krb5kdc[9940](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.194.1@KRBTEST.COM for oryx/127.29.194.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 15:12:13.858223 11463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:13.849031 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:34011 (local address 127.29.194.1:57425)
0807 15:12:13.849244 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0807 15:12:13.849260 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 15:12:13.849327 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:12:13.849523 (+ 196us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:12:13.849523 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:12:13.849733 (+ 210us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:12:13.849790 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:12:13.849792 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:12:13.850606 (+ 814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:12:13.850607 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:12:13.851210 (+ 603us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:12:13.851212 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:12:13.851743 (+ 531us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:12:13.851744 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:12:13.851818 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:12:13.856004 (+ 4186us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:12:13.856010 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:12:13.858121 (+ 2111us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.194.62:34011: Server oryx/127.29.194.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":101,"threads_started":1}
W20260807 15:12:13.858328 11464 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:12:13.849080 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.194.1:57425 (local address 127.29.194.62:34011)
0807 15:12:13.849261 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0807 15:12:13.849263 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:12:13.849332 (+ 69us) server_negotiation.cc:407] Connection header received
0807 15:12:13.849387 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:12:13.849388 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:12:13.849402 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:12:13.849507 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:12:13.849889 (+ 382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:13.850471 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:13.851351 (+ 880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:13.851621 (+ 270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:13.851700 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:12:13.858263 (+ 6563us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.194.1:57425: BlockingRecv error: recv got EOF from 127.29.194.1:57425 (error 108)
Metrics: {"server-negotiator.queue_time_us":146,"thread_start_us":86,"threads_started":1}
Aug 07 15:12:13 dist-test-slave-42z9 krb5kdc[9940](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.194.2@KRBTEST.COM for oryx/127.29.194.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 15:12:13 dist-test-slave-42z9 krb5kdc[9940](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.194.2@KRBTEST.COM for oryx/127.29.194.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 15:12:13.982755 11466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:13.976345 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:34011 (local address 127.29.194.2:33393)
0807 15:12:13.976526 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0807 15:12:13.976539 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 15:12:13.976603 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:12:13.976829 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:12:13.976831 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:12:13.977065 (+ 234us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:12:13.977143 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:12:13.977145 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:12:13.977939 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:12:13.977941 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:12:13.978533 (+ 592us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:12:13.978535 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:12:13.978923 (+ 388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:12:13.978924 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:12:13.979005 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:12:13.979743 (+ 738us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:12:13.979749 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:12:13.982660 (+ 2911us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.194.62:34011: Server oryx/127.29.194.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":74,"threads_started":1}
W20260807 15:12:13.982861 11464 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:12:13.976505 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.194.2:33393 (local address 127.29.194.62:34011)
0807 15:12:13.976570 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0807 15:12:13.976571 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 15:12:13.976584 (+ 13us) server_negotiation.cc:407] Connection header received
0807 15:12:13.976660 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:12:13.976661 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:12:13.976673 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:12:13.976720 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:12:13.977260 (+ 540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:13.977828 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:13.978626 (+ 798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:13.978827 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:13.978897 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:12:13.982798 (+ 3901us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.194.2:33393: BlockingRecv error: recv got EOF from 127.29.194.2:33393 (error 108)
Metrics: {"server-negotiator.queue_time_us":33}
Aug 07 15:12:14 dist-test-slave-42z9 krb5kdc[9940](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.194.3@KRBTEST.COM for oryx/127.29.194.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 15:12:14 dist-test-slave-42z9 krb5kdc[9940](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.194.3@KRBTEST.COM for oryx/127.29.194.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 15:12:14.143546 11469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:14.138390 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:34011 (local address 127.29.194.3:57087)
0807 15:12:14.138555 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0807 15:12:14.138568 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 15:12:14.138634 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:12:14.138810 (+ 176us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:12:14.138812 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:12:14.138991 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:12:14.139046 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:12:14.139047 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:12:14.139880 (+ 833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:12:14.139881 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:12:14.140435 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:12:14.140437 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:12:14.140855 (+ 418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:12:14.140855 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:12:14.141008 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:12:14.141503 (+ 495us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:12:14.141508 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:12:14.143470 (+ 1962us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.194.62:34011: Server oryx/127.29.194.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":96,"threads_started":1}
W20260807 15:12:14.143673 11464 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:12:14.138576 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.194.3:57087 (local address 127.29.194.62:34011)
0807 15:12:14.138638 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0807 15:12:14.138640 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:12:14.138676 (+ 36us) server_negotiation.cc:407] Connection header received
0807 15:12:14.138703 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:12:14.138704 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:12:14.138718 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:12:14.138759 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:12:14.139109 (+ 350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:14.139709 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:14.140529 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:14.140766 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:14.140836 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:12:14.143602 (+ 2766us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.194.3:57087: BlockingRecv error: recv got EOF from 127.29.194.3:57087 (error 108)
Metrics: {"server-negotiator.queue_time_us":34}
Aug 07 15:12:14 dist-test-slave-42z9 krb5kdc[9940](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.194.1@KRBTEST.COM for oryx/127.29.194.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 15:12:14 dist-test-slave-42z9 krb5kdc[9940](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.194.1@KRBTEST.COM for oryx/127.29.194.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 15:12:14.863757 11470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:14.858817 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:34011 (local address 127.29.194.1:38707)
0807 15:12:14.858999 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0807 15:12:14.859011 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 15:12:14.859074 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:12:14.859210 (+ 136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:12:14.859211 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:12:14.859419 (+ 208us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:12:14.859471 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:12:14.859472 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:12:14.860305 (+ 833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:12:14.860306 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:12:14.860866 (+ 560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:12:14.860867 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:12:14.861260 (+ 393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:12:14.861261 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:12:14.861316 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:12:14.861803 (+ 487us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:12:14.861807 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:12:14.863682 (+ 1875us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.194.62:34011: Server oryx/127.29.194.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":91,"threads_started":1}
W20260807 15:12:14.864064 11471 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:12:14.858970 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.194.1:38707 (local address 127.29.194.62:34011)
0807 15:12:14.859084 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0807 15:12:14.859086 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:12:14.859100 (+ 14us) server_negotiation.cc:407] Connection header received
0807 15:12:14.859125 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:12:14.859126 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:12:14.859137 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:12:14.859196 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:12:14.859575 (+ 379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:14.860197 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:14.860960 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:14.861164 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:14.861223 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:12:14.863979 (+ 2756us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.194.1:38707: BlockingRecv error: recv got EOF from 127.29.194.1:38707 (error 108)
Metrics: {"server-negotiator.queue_time_us":93,"thread_start_us":52,"threads_started":1}
Aug 07 15:12:14 dist-test-slave-42z9 krb5kdc[9940](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.194.2@KRBTEST.COM for oryx/127.29.194.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 15:12:14 dist-test-slave-42z9 krb5kdc[9940](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.194.2@KRBTEST.COM for oryx/127.29.194.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 15:12:14.988335 11472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:14.983438 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:34011 (local address 127.29.194.2:44651)
0807 15:12:14.983594 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0807 15:12:14.983607 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 15:12:14.983685 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:12:14.983990 (+ 305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:12:14.983990 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:12:14.984171 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:12:14.984229 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:12:14.984231 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:12:14.984973 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:12:14.984975 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:12:14.985588 (+ 613us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:12:14.985591 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:12:14.985931 (+ 340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:12:14.985932 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:12:14.985980 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:12:14.986466 (+ 486us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:12:14.986472 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:12:14.988260 (+ 1788us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.194.62:34011: Server oryx/127.29.194.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":64,"threads_started":1}
W20260807 15:12:14.988443 11471 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:12:14.983594 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.194.2:44651 (local address 127.29.194.62:34011)
0807 15:12:14.983663 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0807 15:12:14.983666 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:12:14.983677 (+ 11us) server_negotiation.cc:407] Connection header received
0807 15:12:14.983740 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:12:14.983741 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:12:14.983842 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:12:14.983957 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:12:14.984332 (+ 375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:14.984855 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:14.985668 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:14.985869 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:14.985929 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:12:14.988377 (+ 2448us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.194.2:44651: BlockingRecv error: recv got EOF from 127.29.194.2:44651 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
Aug 07 15:12:15 dist-test-slave-42z9 krb5kdc[9940](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.194.3@KRBTEST.COM for oryx/127.29.194.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 15:12:15 dist-test-slave-42z9 krb5kdc[9940](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.194.3@KRBTEST.COM for oryx/127.29.194.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 15:12:15.150115 11473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:15.144188 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:34011 (local address 127.29.194.3:38373)
0807 15:12:15.144393 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0807 15:12:15.144408 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 15:12:15.144498 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:12:15.144778 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:12:15.144780 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:12:15.145007 (+ 227us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:12:15.145060 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:12:15.145061 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:12:15.145829 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:12:15.145831 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:12:15.146613 (+ 782us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:12:15.146617 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:12:15.147012 (+ 395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:12:15.147014 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:12:15.147067 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:12:15.147567 (+ 500us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:12:15.147571 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:12:15.150019 (+ 2448us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.194.62:34011: Server oryx/127.29.194.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":102,"threads_started":1}
W20260807 15:12:15.150254 11471 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:12:15.144345 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.194.3:38373 (local address 127.29.194.62:34011)
0807 15:12:15.144415 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0807 15:12:15.144417 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:12:15.144544 (+ 127us) server_negotiation.cc:407] Connection header received
0807 15:12:15.144592 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:12:15.144593 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:12:15.144620 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:12:15.144706 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:12:15.145152 (+ 446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:15.145731 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:15.146714 (+ 983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:15.146912 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:15.146974 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:12:15.150182 (+ 3208us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.194.3:38373: BlockingRecv error: recv got EOF from 127.29.194.3:38373 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
Aug 07 15:12:15 dist-test-slave-42z9 krb5kdc[9940](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.194.1@KRBTEST.COM for oryx/127.29.194.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 15:12:15 dist-test-slave-42z9 krb5kdc[9940](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.194.1@KRBTEST.COM for oryx/127.29.194.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 15:12:15.870265 11474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:15.864428 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:34011 (local address 127.29.194.1:56157)
0807 15:12:15.864560 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0807 15:12:15.864573 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 15:12:15.864626 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:12:15.864873 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:12:15.864874 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:12:15.865059 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:12:15.865109 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:12:15.865110 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:12:15.866255 (+ 1145us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:12:15.866257 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:12:15.866879 (+ 622us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:12:15.866881 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:12:15.867388 (+ 507us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:12:15.867389 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:12:15.867445 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:12:15.868139 (+ 694us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:12:15.868146 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:12:15.870204 (+ 2058us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.194.62:34011: Server oryx/127.29.194.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":78,"threads_started":1}
W20260807 15:12:15.870460 11476 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:12:15.864562 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.194.1:56157 (local address 127.29.194.62:34011)
0807 15:12:15.864727 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0807 15:12:15.864729 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:12:15.864740 (+ 11us) server_negotiation.cc:407] Connection header received
0807 15:12:15.864767 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:12:15.864768 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:12:15.864781 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:12:15.864861 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:12:15.865276 (+ 415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:15.866094 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:15.867033 (+ 939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:15.867278 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:15.867354 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:12:15.870382 (+ 3028us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.194.1:56157: BlockingRecv error: recv got EOF from 127.29.194.1:56157 (error 108)
Metrics: {"server-negotiator.queue_time_us":143,"thread_start_us":73,"threads_started":1}
Aug 07 15:12:15 dist-test-slave-42z9 krb5kdc[9940](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.194.2@KRBTEST.COM for oryx/127.29.194.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 15:12:15 dist-test-slave-42z9 krb5kdc[9940](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.194.2@KRBTEST.COM for oryx/127.29.194.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 15:12:15.994776 11477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:15.989081 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:34011 (local address 127.29.194.2:42977)
0807 15:12:15.989325 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0807 15:12:15.989341 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 15:12:15.989429 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:12:15.989632 (+ 203us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:12:15.989633 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:12:15.989818 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:12:15.989871 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:12:15.989872 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:12:15.990522 (+ 650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:12:15.990524 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:12:15.991057 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:12:15.991058 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:12:15.991570 (+ 512us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:12:15.991571 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:12:15.991618 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:12:15.992331 (+ 713us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:12:15.992339 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:12:15.994678 (+ 2339us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.194.62:34011: Server oryx/127.29.194.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":211,"thread_start_us":110,"threads_started":1}
W20260807 15:12:15.994903 11476 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:12:15.989258 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.194.2:42977 (local address 127.29.194.62:34011)
0807 15:12:15.989397 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0807 15:12:15.989399 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:12:15.989418 (+ 19us) server_negotiation.cc:407] Connection header received
0807 15:12:15.989524 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:12:15.989525 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:12:15.989546 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:12:15.989616 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:12:15.989924 (+ 308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:15.990431 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:15.991203 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:15.991470 (+ 267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:15.991563 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:12:15.994825 (+ 3262us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.194.2:42977: BlockingRecv error: recv got EOF from 127.29.194.2:42977 (error 108)
Metrics: {"server-negotiator.queue_time_us":101}
Aug 07 15:12:16 dist-test-slave-42z9 krb5kdc[9940](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.194.3@KRBTEST.COM for oryx/127.29.194.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 15:12:16 dist-test-slave-42z9 krb5kdc[9940](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.194.3@KRBTEST.COM for oryx/127.29.194.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 15:12:16.155588 11479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:16.150639 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:34011 (local address 127.29.194.3:60859)
0807 15:12:16.150787 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0807 15:12:16.150796 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0807 15:12:16.150852 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:12:16.150981 (+ 129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:12:16.150982 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:12:16.151163 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:12:16.151212 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:12:16.151213 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:12:16.152050 (+ 837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:12:16.152051 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:12:16.152629 (+ 578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:12:16.152631 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:12:16.153064 (+ 433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:12:16.153066 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:12:16.153118 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:12:16.153682 (+ 564us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:12:16.153686 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:12:16.155521 (+ 1835us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.194.62:34011: Server oryx/127.29.194.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":85,"threads_started":1}
W20260807 15:12:16.155717 11476 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:12:16.150741 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.194.3:60859 (local address 127.29.194.62:34011)
0807 15:12:16.150795 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0807 15:12:16.150797 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:12:16.150856 (+ 59us) server_negotiation.cc:407] Connection header received
0807 15:12:16.150895 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:12:16.150896 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:12:16.150908 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:12:16.150971 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:12:16.151334 (+ 363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:16.151923 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:16.152747 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:16.152964 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:16.153035 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:12:16.155650 (+ 2615us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.194.3:60859: BlockingRecv error: recv got EOF from 127.29.194.3:60859 (error 108)
Metrics: {"server-negotiator.queue_time_us":30}
Aug 07 15:12:16 dist-test-slave-42z9 krb5kdc[9940](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.194.1@KRBTEST.COM for oryx/127.29.194.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 15:12:16 dist-test-slave-42z9 krb5kdc[9940](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.194.1@KRBTEST.COM for oryx/127.29.194.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 15:12:16.880265 11480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:16.870933 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:34011 (local address 127.29.194.1:33623)
0807 15:12:16.871126 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0807 15:12:16.871137 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 15:12:16.871189 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:12:16.871475 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:12:16.871478 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:12:16.871755 (+ 277us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:12:16.871842 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:12:16.871845 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:12:16.872821 (+ 976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:12:16.872822 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:12:16.873426 (+ 604us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:12:16.873428 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:12:16.873816 (+ 388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:12:16.873817 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:12:16.873857 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:12:16.877724 (+ 3867us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:12:16.877730 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:12:16.880155 (+ 2425us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.194.62:34011: Server oryx/127.29.194.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":94,"threads_started":1}
W20260807 15:12:16.880429 11482 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:12:16.871035 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.194.1:33623 (local address 127.29.194.62:34011)
0807 15:12:16.871160 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0807 15:12:16.871161 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 15:12:16.871175 (+ 14us) server_negotiation.cc:407] Connection header received
0807 15:12:16.871264 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:12:16.871265 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:12:16.871277 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:12:16.871345 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:12:16.871986 (+ 641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:16.872710 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:16.873547 (+ 837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:16.873756 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:16.873814 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:12:16.880311 (+ 6497us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.194.1:33623: BlockingRecv error: recv got EOF from 127.29.194.1:33623 (error 108)
Metrics: {"server-negotiator.queue_time_us":101,"thread_start_us":63,"threads_started":1}
Aug 07 15:12:16 dist-test-slave-42z9 krb5kdc[9940](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.194.2@KRBTEST.COM for oryx/127.29.194.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 15:12:16 dist-test-slave-42z9 krb5kdc[9940](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.194.2@KRBTEST.COM for oryx/127.29.194.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 15:12:17.002372 11483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:16.995562 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:34011 (local address 127.29.194.2:47507)
0807 15:12:16.995784 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0807 15:12:16.995802 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 15:12:16.995897 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:12:16.996087 (+ 190us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:12:16.996088 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:12:16.996305 (+ 217us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:12:16.996372 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:12:16.996376 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:12:16.997226 (+ 850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:12:16.997229 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:12:16.997986 (+ 757us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:12:16.997990 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:12:16.998521 (+ 531us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:12:16.998522 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:12:16.998599 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:12:16.999122 (+ 523us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:12:16.999127 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:12:17.002256 (+ 3129us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.194.62:34011: Server oryx/127.29.194.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":87,"threads_started":1}
W20260807 15:12:17.002592 11482 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:12:16.995825 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.194.2:47507 (local address 127.29.194.62:34011)
0807 15:12:16.995949 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0807 15:12:16.995951 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:12:16.995959 (+ 8us) server_negotiation.cc:407] Connection header received
0807 15:12:16.995994 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:12:16.995995 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:12:16.996007 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:12:16.996066 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:12:16.996500 (+ 434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:16.997062 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:16.998133 (+ 1071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:16.998362 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:16.998438 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:12:17.002518 (+ 4080us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.194.2:47507: BlockingRecv error: recv got EOF from 127.29.194.2:47507 (error 108)
Metrics: {"server-negotiator.queue_time_us":95}
Aug 07 15:12:17 dist-test-slave-42z9 krb5kdc[9940](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.194.3@KRBTEST.COM for oryx/127.29.194.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 15:12:17 dist-test-slave-42z9 krb5kdc[9940](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.194.3@KRBTEST.COM for oryx/127.29.194.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 15:12:17.162067 11484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:17.156101 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:34011 (local address 127.29.194.3:41809)
0807 15:12:17.156231 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0807 15:12:17.156241 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 15:12:17.156312 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:12:17.156437 (+ 125us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:12:17.156438 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:12:17.156658 (+ 220us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:12:17.156704 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:12:17.156706 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:12:17.157344 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:12:17.157345 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:12:17.157950 (+ 605us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:12:17.157953 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:12:17.158355 (+ 402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:12:17.158356 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:12:17.158428 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:12:17.159090 (+ 662us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:12:17.159103 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:12:17.161966 (+ 2863us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.194.62:34011: Server oryx/127.29.194.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":77,"threads_started":1}
W20260807 15:12:17.162166 11482 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:12:17.156158 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.194.3:41809 (local address 127.29.194.62:34011)
0807 15:12:17.156204 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0807 15:12:17.156206 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:12:17.156301 (+ 95us) server_negotiation.cc:407] Connection header received
0807 15:12:17.156371 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:12:17.156371 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:12:17.156386 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:12:17.156432 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:12:17.156774 (+ 342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:17.157274 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:17.158025 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:17.158222 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:17.158332 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:12:17.162108 (+ 3776us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.194.3:41809: BlockingRecv error: recv got EOF from 127.29.194.3:41809 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
Aug 07 15:12:17 dist-test-slave-42z9 krb5kdc[9940](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.194.1@KRBTEST.COM for oryx/127.29.194.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 15:12:17 dist-test-slave-42z9 krb5kdc[9940](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.194.1@KRBTEST.COM for oryx/127.29.194.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 15:12:17.886507 11488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:17.880980 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:34011 (local address 127.29.194.1:47511)
0807 15:12:17.881129 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0807 15:12:17.881141 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 15:12:17.881202 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:12:17.881499 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:12:17.881502 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:12:17.881939 (+ 437us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:12:17.882008 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:12:17.882012 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:12:17.882769 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:12:17.882774 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:12:17.883314 (+ 540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:12:17.883316 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:12:17.883702 (+ 386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:12:17.883703 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:12:17.883754 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:12:17.884265 (+ 511us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:12:17.884270 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:12:17.886444 (+ 2174us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.194.62:34011: Server oryx/127.29.194.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":69,"threads_started":1}
W20260807 15:12:17.886636 11489 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:12:17.881107 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.194.1:47511 (local address 127.29.194.62:34011)
0807 15:12:17.881223 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0807 15:12:17.881225 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:12:17.881234 (+ 9us) server_negotiation.cc:407] Connection header received
0807 15:12:17.881264 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:12:17.881265 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:12:17.881277 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:12:17.881335 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:12:17.882131 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:17.882673 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:17.883416 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:17.883614 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:17.883675 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:12:17.886559 (+ 2884us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.194.1:47511: BlockingRecv error: recv got EOF from 127.29.194.1:47511 (error 108)
Metrics: {"server-negotiator.queue_time_us":95,"thread_start_us":50,"threads_started":1}
Aug 07 15:12:18 dist-test-slave-42z9 krb5kdc[9940](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.194.2@KRBTEST.COM for oryx/127.29.194.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 15:12:18 dist-test-slave-42z9 krb5kdc[9940](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.194.2@KRBTEST.COM for oryx/127.29.194.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 15:12:18.014775 11491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:18.003106 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:34011 (local address 127.29.194.2:43537)
0807 15:12:18.003235 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0807 15:12:18.003247 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 15:12:18.003307 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:12:18.003448 (+ 141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:12:18.003448 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:12:18.003669 (+ 221us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:12:18.003722 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:12:18.003724 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:12:18.004411 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:12:18.004412 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:12:18.005110 (+ 698us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:12:18.005113 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:12:18.005596 (+ 483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:12:18.005599 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:12:18.005687 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:12:18.006267 (+ 580us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:12:18.006272 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:12:18.014686 (+ 8414us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.194.62:34011: Server oryx/127.29.194.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":78,"threads_started":1}
W20260807 15:12:18.014889 11489 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:12:18.003209 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.194.2:43537 (local address 127.29.194.62:34011)
0807 15:12:18.003302 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0807 15:12:18.003304 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:12:18.003349 (+ 45us) server_negotiation.cc:407] Connection header received
0807 15:12:18.003379 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:12:18.003380 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:12:18.003391 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:12:18.003434 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:12:18.003784 (+ 350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:18.004309 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:18.005178 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:18.005374 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:18.005431 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:12:18.014822 (+ 9391us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.194.2:43537: BlockingRecv error: recv got EOF from 127.29.194.2:43537 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
Aug 07 15:12:18 dist-test-slave-42z9 krb5kdc[9940](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.194.3@KRBTEST.COM for oryx/127.29.194.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 15:12:18 dist-test-slave-42z9 krb5kdc[9940](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.194.3@KRBTEST.COM for oryx/127.29.194.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 15:12:18.169168 11492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:18.162843 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:34011 (local address 127.29.194.3:56737)
0807 15:12:18.163119 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0807 15:12:18.163136 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 15:12:18.163220 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:12:18.163417 (+ 197us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:12:18.163418 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:12:18.163630 (+ 212us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:12:18.163690 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:12:18.163692 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:12:18.164606 (+ 914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:12:18.164608 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:12:18.165367 (+ 759us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:12:18.165372 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:12:18.165890 (+ 518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:12:18.165892 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:12:18.165957 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:12:18.166604 (+ 647us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:12:18.166611 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:12:18.169102 (+ 2491us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.194.62:34011: Server oryx/127.29.194.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":241,"thread_start_us":140,"threads_started":1}
W20260807 15:12:18.169395 11489 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:12:18.163025 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.194.3:56737 (local address 127.29.194.62:34011)
0807 15:12:18.163192 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0807 15:12:18.163194 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:12:18.163213 (+ 19us) server_negotiation.cc:407] Connection header received
0807 15:12:18.163304 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:12:18.163306 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:12:18.163323 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:12:18.163393 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:12:18.163836 (+ 443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:18.164508 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:18.165544 (+ 1036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:18.165771 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:18.165851 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:12:18.169305 (+ 3454us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.194.3:56737: BlockingRecv error: recv got EOF from 127.29.194.3:56737 (error 108)
Metrics: {"server-negotiator.queue_time_us":88}
Aug 07 15:12:18 dist-test-slave-42z9 krb5kdc[9940](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.194.1@KRBTEST.COM for oryx/127.29.194.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 15:12:18 dist-test-slave-42z9 krb5kdc[9940](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.194.1@KRBTEST.COM for oryx/127.29.194.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 15:12:18.892189 11494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:18.887096 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:34011 (local address 127.29.194.1:59559)
0807 15:12:18.887219 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0807 15:12:18.887230 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 15:12:18.887286 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:12:18.887433 (+ 147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:12:18.887434 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:12:18.887600 (+ 166us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:12:18.887653 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:12:18.887654 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:12:18.888381 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:12:18.888383 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:12:18.889052 (+ 669us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:12:18.889054 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:12:18.889636 (+ 582us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:12:18.889637 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:12:18.889707 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:12:18.890195 (+ 488us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:12:18.890200 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:12:18.892113 (+ 1913us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.194.62:34011: Server oryx/127.29.194.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":65,"threads_started":1}
W20260807 15:12:18.892314 11495 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:12:18.887201 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.194.1:59559 (local address 127.29.194.62:34011)
0807 15:12:18.887316 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0807 15:12:18.887318 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:12:18.887326 (+ 8us) server_negotiation.cc:407] Connection header received
0807 15:12:18.887349 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:12:18.887350 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:12:18.887360 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:12:18.887417 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:12:18.887708 (+ 291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:18.888226 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:18.889186 (+ 960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:18.889529 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:18.889624 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:12:18.892243 (+ 2619us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.194.1:59559: BlockingRecv error: recv got EOF from 127.29.194.1:59559 (error 108)
Metrics: {"server-negotiator.queue_time_us":94,"thread_start_us":54,"threads_started":1}
Aug 07 15:12:19 dist-test-slave-42z9 krb5kdc[9940](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.194.2@KRBTEST.COM for oryx/127.29.194.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 15:12:19 dist-test-slave-42z9 krb5kdc[9940](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.194.2@KRBTEST.COM for oryx/127.29.194.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 15:12:19.020949 11496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:19.015449 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:34011 (local address 127.29.194.2:45791)
0807 15:12:19.015625 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0807 15:12:19.015641 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 15:12:19.015724 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:12:19.015856 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:12:19.015857 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:12:19.016061 (+ 204us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:12:19.016115 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:12:19.016117 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:12:19.016772 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:12:19.016772 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:12:19.017398 (+ 626us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:12:19.017401 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:12:19.017838 (+ 437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:12:19.017839 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:12:19.017895 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:12:19.018392 (+ 497us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:12:19.018397 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:12:19.020864 (+ 2467us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.194.62:34011: Server oryx/127.29.194.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":91,"threads_started":1}
W20260807 15:12:19.021029 11495 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:12:19.015594 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.194.2:45791 (local address 127.29.194.62:34011)
0807 15:12:19.015656 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0807 15:12:19.015659 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:12:19.015704 (+ 45us) server_negotiation.cc:407] Connection header received
0807 15:12:19.015783 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:12:19.015784 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:12:19.015797 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:12:19.015846 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:12:19.016191 (+ 345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:19.016690 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:19.017504 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:19.017749 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:19.017814 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:12:19.020969 (+ 3155us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.194.2:45791: BlockingRecv error: recv got EOF from 127.29.194.2:45791 (error 108)
Metrics: {"server-negotiator.queue_time_us":34}
I20260807 15:12:19.072599 30472 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7fZSIL/build/release/bin/kudu with pid 10802
I20260807 15:12:19.092903 30472 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7fZSIL/build/release/bin/kudu with pid 11021
I20260807 15:12:19.104051 30472 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7fZSIL/build/release/bin/kudu with pid 11215
I20260807 15:12:19.121249 30472 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7fZSIL/build/release/bin/kudu with pid 10703
2026-08-07T15:12:19Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (17776 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-07T15:12:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T15:12:19Z Disabled control of system clock
I20260807 15:12:19.167047 30472 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7fZSIL/build/release/bin/kudu
/tmp/dist-test-task7fZSIL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786115480851507-30472-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786115480851507-30472-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786115480851507-30472-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786115480851507-30472-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.194.62:41515
--webserver_interface=127.29.194.62
--webserver_port=0
--builtin_ntp_servers=127.29.194.20:35849
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.194.62:41515
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260807 15:12:19.263381 11501 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:12:19.263634 11501 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:12:19.263657 11501 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:12:19.265224 11501 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:12:19.265307 11501 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:12:19.265321 11501 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:12:19.265332 11501 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:12:19.265344 11501 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:12:19.267035 11501 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.194.20:35849
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786115480851507-30472-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786115480851507-30472-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.194.62:41515
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.194.62:41515
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786115480851507-30472-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.194.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11501
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786115480851507-30472-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:12:19.267341 11501 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:12:19.267707 11501 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:12:19.273275 11510 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:12:19.273419 11513 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:12:19.273764 11501 server_base.cc:1034] running on GCE node
W20260807 15:12:19.273519 11509 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:12:19.274150 11501 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:12:19.274502 11501 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:12:19.275715 11501 hybrid_clock.cc:648] HybridClock initialized: now 1786115539275698 us; error 38 us; skew 500 ppm
I20260807 15:12:19.277581 11501 webserver.cc:460] Webserver started at http://127.29.194.62:35299/ using document root <none> and password file <none>
I20260807 15:12:19.277961 11501 fs_manager.cc:359] Metadata directory not provided
I20260807 15:12:19.278025 11501 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:12:19.278270 11501 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:12:19.279594 11501 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786115480851507-30472-0/minicluster-data/master-0/data/instance:
uuid: "4b9a6e885c1e49689438368eaeeb1eb6"
format_stamp: "Formatted at 2026-08-07 15:12:19 on dist-test-slave-42z9"
server_key: "ef931978e7216578e6db92be87f1a531"
server_key_iv: "eb44a09602a2d28ef31fd34e19461ebb"
server_key_version: "encryptionkey@0"
I20260807 15:12:19.279949 11501 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786115480851507-30472-0/minicluster-data/master-0/wal/instance:
uuid: "4b9a6e885c1e49689438368eaeeb1eb6"
format_stamp: "Formatted at 2026-08-07 15:12:19 on dist-test-slave-42z9"
server_key: "ef931978e7216578e6db92be87f1a531"
server_key_iv: "eb44a09602a2d28ef31fd34e19461ebb"
server_key_version: "encryptionkey@0"
I20260807 15:12:19.281692 11501 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.004s
I20260807 15:12:19.282800 11523 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:12:19.283066 11501 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:12:19.283167 11501 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786115480851507-30472-0/minicluster-data/master-0/data,/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786115480851507-30472-0/minicluster-data/master-0/wal
uuid: "4b9a6e885c1e49689438368eaeeb1eb6"
format_stamp: "Formatted at 2026-08-07 15:12:19 on dist-test-slave-42z9"
server_key: "ef931978e7216578e6db92be87f1a531"
server_key_iv: "eb44a09602a2d28ef31fd34e19461ebb"
server_key_version: "encryptionkey@0"
I20260807 15:12:19.283257 11501 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786115480851507-30472-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786115480851507-30472-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786115480851507-30472-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:12:19.291726 11501 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:12:19.292078 11501 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:12:19.292212 11501 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:12:19.296504 11501 rpc_server.cc:307] RPC server started. Bound to: 127.29.194.62:41515
I20260807 15:12:19.296556 11598 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.194.62:41515 every 8 connection(s)
I20260807 15:12:19.296890 11501 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786115480851507-30472-0/minicluster-data/master-0/data/info.pb
I20260807 15:12:19.297703 11599 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:12:19.300482 11599 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4b9a6e885c1e49689438368eaeeb1eb6: Bootstrap starting.
I20260807 15:12:19.301121 11599 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4b9a6e885c1e49689438368eaeeb1eb6: Neither blocks nor log segments found. Creating new log.
I20260807 15:12:19.301409 11599 log.cc:824] T 00000000000000000000000000000000 P 4b9a6e885c1e49689438368eaeeb1eb6: Log is configured to *not* fsync() on all Append() calls
I20260807 15:12:19.302232 11599 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4b9a6e885c1e49689438368eaeeb1eb6: No bootstrap required, opened a new log
I20260807 15:12:19.303932 11599 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4b9a6e885c1e49689438368eaeeb1eb6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b9a6e885c1e49689438368eaeeb1eb6" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 41515 } }
I20260807 15:12:19.304101 11599 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4b9a6e885c1e49689438368eaeeb1eb6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:12:19.304124 11599 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4b9a6e885c1e49689438368eaeeb1eb6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b9a6e885c1e49689438368eaeeb1eb6, State: Initialized, Role: FOLLOWER
I20260807 15:12:19.304263 11599 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4b9a6e885c1e49689438368eaeeb1eb6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b9a6e885c1e49689438368eaeeb1eb6" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 41515 } }
I20260807 15:12:19.304343 11599 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4b9a6e885c1e49689438368eaeeb1eb6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:12:19.304376 11599 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4b9a6e885c1e49689438368eaeeb1eb6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:12:19.304419 11599 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4b9a6e885c1e49689438368eaeeb1eb6 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:12:19.305051 11599 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4b9a6e885c1e49689438368eaeeb1eb6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b9a6e885c1e49689438368eaeeb1eb6" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 41515 } }
I20260807 15:12:19.305186 11599 leader_election.cc:302] T 00000000000000000000000000000000 P 4b9a6e885c1e49689438368eaeeb1eb6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4b9a6e885c1e49689438368eaeeb1eb6; no voters:
I20260807 15:12:19.305363 11599 leader_election.cc:288] T 00000000000000000000000000000000 P 4b9a6e885c1e49689438368eaeeb1eb6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 15:12:19.305415 11605 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4b9a6e885c1e49689438368eaeeb1eb6 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:12:19.305423 30472 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7fZSIL/build/release/bin/kudu as pid 11501
I20260807 15:12:19.305548 30472 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786115480851507-30472-0/minicluster-data/master-0/wal/instance
I20260807 15:12:19.305670 11605 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4b9a6e885c1e49689438368eaeeb1eb6 [term 1 LEADER]: Becoming Leader. State: Replica: 4b9a6e885c1e49689438368eaeeb1eb6, State: Running, Role: LEADER
I20260807 15:12:19.305706 11599 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4b9a6e885c1e49689438368eaeeb1eb6 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:12:19.305749 30472 external_mini_cluster.cc:1428] Setting key c5b93352cd0b4f52ccf1b894addb8f1b
I20260807 15:12:19.305799 11605 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4b9a6e885c1e49689438368eaeeb1eb6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b9a6e885c1e49689438368eaeeb1eb6" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 41515 } }
I20260807 15:12:19.306411 11606 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4b9a6e885c1e49689438368eaeeb1eb6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4b9a6e885c1e49689438368eaeeb1eb6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b9a6e885c1e49689438368eaeeb1eb6" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 41515 } } }
I20260807 15:12:19.306506 11606 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4b9a6e885c1e49689438368eaeeb1eb6 [sys.catalog]: This master's current role is: LEADER
I20260807 15:12:19.306666 11608 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4b9a6e885c1e49689438368eaeeb1eb6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4b9a6e885c1e49689438368eaeeb1eb6. Latest consensus state: current_term: 1 leader_uuid: "4b9a6e885c1e49689438368eaeeb1eb6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b9a6e885c1e49689438368eaeeb1eb6" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 41515 } } }
I20260807 15:12:19.306728 11608 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4b9a6e885c1e49689438368eaeeb1eb6 [sys.catalog]: This master's current role is: LEADER
I20260807 15:12:19.306720 11627 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:12:19.307427 11627 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:12:19.309231 11627 catalog_manager.cc:1347] Generated new cluster ID: daf73d5a4e17492c87b176baa6ed0f39
I20260807 15:12:19.309293 11627 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:12:19.311807 11613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:19.306244 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34928 (local address 127.29.194.62:41515)
0807 15:12:19.307011 (+ 767us) server_negotiation.cc:206] Beginning negotiation
0807 15:12:19.307021 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 15:12:19.307062 (+ 41us) server_negotiation.cc:407] Connection header received
0807 15:12:19.307197 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:12:19.307204 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:12:19.307280 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:12:19.307464 (+ 184us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:12:19.307716 (+ 252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:19.308996 (+ 1280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:19.309926 (+ 930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:19.310363 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:19.310476 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:12:19.311171 (+ 695us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:12:19.311175 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:12:19.311176 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:12:19.311348 (+ 172us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:12:19.311386 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:12:19.311397 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:12:19.311457 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0807 15:12:19.311546 (+ 89us) server_negotiation.cc:299] Negotiation successful
0807 15:12:19.311630 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":666,"thread_start_us":72,"threads_started":1}
I20260807 15:12:19.332908 11627 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:12:19.333725 11627 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:12:19.345850 11627 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4b9a6e885c1e49689438368eaeeb1eb6: Generated new TSK 0
I20260807 15:12:19.346051 11627 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 15:12:19.347694 30472 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7fZSIL/build/release/bin/kudu
/tmp/dist-test-task7fZSIL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786115480851507-30472-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786115480851507-30472-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786115480851507-30472-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786115480851507-30472-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.194.1:0
--local_ip_for_outbound_sockets=127.29.194.1
--webserver_interface=127.29.194.1
--webserver_port=0
--tserver_master_addrs=127.29.194.62:41515
--builtin_ntp_servers=127.29.194.20:35849
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260807 15:12:19.437357 11644 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:12:19.437649 11644 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:12:19.437682 11644 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:12:19.438925 11644 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:12:19.438977 11644 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:12:19.439011 11644 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.194.1
I20260807 15:12:19.442030 11644 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.194.20:35849
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786115480851507-30472-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786115480851507-30472-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.194.1:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786115480851507-30472-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.194.1
--webserver_port=0
--tserver_master_addrs=127.29.194.62:41515
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11644
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.194.1
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786115480851507-30472-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:12:19.442437 11644 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:12:19.442847 11644 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:12:19.449154 11654 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:12:19.449154 11644 server_base.cc:1034] running on GCE node
W20260807 15:12:19.449172 11657 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:12:19.449312 11653 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:12:19.450016 11644 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:12:19.450352 11644 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:12:19.451602 11644 hybrid_clock.cc:648] HybridClock initialized: now 1786115539451552 us; error 70 us; skew 500 ppm
I20260807 15:12:19.453130 11644 webserver.cc:460] Webserver started at http://127.29.194.1:38293/ using document root <none> and password file <none>
I20260807 15:12:19.453385 11644 fs_manager.cc:359] Metadata directory not provided
I20260807 15:12:19.453435 11644 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:12:19.453622 11644 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:12:19.454793 11644 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786115480851507-30472-0/minicluster-data/ts-0/data/instance:
uuid: "78a5aada82394bea9a23c488b6d66c84"
format_stamp: "Formatted at 2026-08-07 15:12:19 on dist-test-slave-42z9"
server_key: "0f1804bd2aadbd2ff28e0bc818a2e8f5"
server_key_iv: "97b0264de24dab84cc8de5b0dfe7e8af"
server_key_version: "encryptionkey@0"
I20260807 15:12:19.455137 11644 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786115480851507-30472-0/minicluster-data/ts-0/wal/instance:
uuid: "78a5aada82394bea9a23c488b6d66c84"
format_stamp: "Formatted at 2026-08-07 15:12:19 on dist-test-slave-42z9"
server_key: "0f1804bd2aadbd2ff28e0bc818a2e8f5"
server_key_iv: "97b0264de24dab84cc8de5b0dfe7e8af"
server_key_version: "encryptionkey@0"
I20260807 15:12:19.456476 11644 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 15:12:19.457302 11665 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:12:19.457530 11644 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:12:19.457626 11644 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786115480851507-30472-0/minicluster-data/ts-0/data,/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786115480851507-30472-0/minicluster-data/ts-0/wal
uuid: "78a5aada82394bea9a23c488b6d66c84"
format_stamp: "Formatted at 2026-08-07 15:12:19 on dist-test-slave-42z9"
server_key: "0f1804bd2aadbd2ff28e0bc818a2e8f5"
server_key_iv: "97b0264de24dab84cc8de5b0dfe7e8af"
server_key_version: "encryptionkey@0"
I20260807 15:12:19.457688 11644 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786115480851507-30472-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786115480851507-30472-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786115480851507-30472-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:12:19.467669 11644 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:12:19.467972 11644 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:12:19.468086 11644 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:12:19.468298 11644 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:12:19.468681 11644 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:12:19.468721 11644 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:12:19.468744 11644 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:12:19.468760 11644 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:12:19.474588 11644 rpc_server.cc:307] RPC server started. Bound to: 127.29.194.1:43859
I20260807 15:12:19.474663 11830 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.194.1:43859 every 8 connection(s)
I20260807 15:12:19.474989 11644 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786115480851507-30472-0/minicluster-data/ts-0/data/info.pb
I20260807 15:12:19.480559 11613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:19.475734 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.194.1:50653 (local address 127.29.194.62:41515)
0807 15:12:19.475793 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0807 15:12:19.475795 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:12:19.476034 (+ 239us) server_negotiation.cc:407] Connection header received
0807 15:12:19.476548 (+ 514us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:12:19.476549 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:12:19.476561 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:12:19.476624 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:12:19.477294 (+ 670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:19.477899 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:19.478942 (+ 1043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:19.479139 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:19.479211 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:12:19.480151 (+ 940us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:12:19.480152 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:12:19.480152 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:12:19.480157 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:12:19.480173 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:12:19.480177 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:12:19.480227 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0807 15:12:19.480402 (+ 175us) server_negotiation.cc:299] Negotiation successful
0807 15:12:19.480470 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260807 15:12:19.480608 11833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:19.475698 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:41515 (local address 127.29.194.1:50653)
0807 15:12:19.475903 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0807 15:12:19.475946 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0807 15:12:19.476432 (+ 486us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:12:19.476686 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:12:19.476689 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:12:19.476698 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:12:19.477191 (+ 493us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:12:19.477195 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:12:19.478003 (+ 808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:12:19.478005 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:12:19.478832 (+ 827us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:12:19.478834 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:12:19.479276 (+ 442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:12:19.479278 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:12:19.479431 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:12:19.479998 (+ 567us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 15:12:19.479999 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:12:19.480030 (+ 31us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 15:12:19.480055 (+ 25us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:12:19.480239 (+ 184us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:12:19.480242 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:12:19.480244 (+ 2us) client_negotiation.cc:769] Sending connection context
0807 15:12:19.480294 (+ 50us) client_negotiation.cc:240] Negotiation successful
0807 15:12:19.480384 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":89,"threads_started":1}
I20260807 15:12:19.481542 11831 heartbeater.cc:347] Connected to a master server at 127.29.194.62:41515
I20260807 15:12:19.481658 11831 heartbeater.cc:464] Registering TS with master...
I20260807 15:12:19.481864 11831 heartbeater.cc:511] Master 127.29.194.62:41515 requested a full tablet report, sending...
I20260807 15:12:19.482714 11547 ts_manager.cc:194] Registered new tserver with Master: 78a5aada82394bea9a23c488b6d66c84 (127.29.194.1:43859)
I20260807 15:12:19.483510 11547 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.29.194.1:50653
I20260807 15:12:19.484091 30472 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7fZSIL/build/release/bin/kudu as pid 11644
I20260807 15:12:19.484212 30472 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786115480851507-30472-0/minicluster-data/ts-0/wal/instance
I20260807 15:12:19.484449 30472 external_mini_cluster.cc:1428] Setting key 25322e9700879705d8a421e23288c2df
I20260807 15:12:19.485777 30472 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7fZSIL/build/release/bin/kudu
/tmp/dist-test-task7fZSIL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786115480851507-30472-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786115480851507-30472-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786115480851507-30472-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786115480851507-30472-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.194.2:0
--local_ip_for_outbound_sockets=127.29.194.2
--webserver_interface=127.29.194.2
--webserver_port=0
--tserver_master_addrs=127.29.194.62:41515
--builtin_ntp_servers=127.29.194.20:35849
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260807 15:12:19.574013 11834 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:12:19.574292 11834 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:12:19.574318 11834 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:12:19.575908 11834 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:12:19.575981 11834 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:12:19.576015 11834 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.194.2
I20260807 15:12:19.578197 11834 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.194.20:35849
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786115480851507-30472-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786115480851507-30472-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.194.2:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786115480851507-30472-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.194.2
--webserver_port=0
--tserver_master_addrs=127.29.194.62:41515
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11834
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.194.2
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786115480851507-30472-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:12:19.578588 11834 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:12:19.578956 11834 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 15:12:19.585027 11834 server_base.cc:1034] running on GCE node
W20260807 15:12:19.584991 11844 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:12:19.585047 11840 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:12:19.585001 11839 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:12:19.585760 11834 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:12:19.586058 11834 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:12:19.587270 11834 hybrid_clock.cc:648] HybridClock initialized: now 1786115539587235 us; error 52 us; skew 500 ppm
I20260807 15:12:19.589309 11834 webserver.cc:460] Webserver started at http://127.29.194.2:45945/ using document root <none> and password file <none>
I20260807 15:12:19.589684 11834 fs_manager.cc:359] Metadata directory not provided
I20260807 15:12:19.589746 11834 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:12:19.589905 11834 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:12:19.591419 11834 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786115480851507-30472-0/minicluster-data/ts-1/data/instance:
uuid: "2c70de56c780438c8ef2c1dffcde7d90"
format_stamp: "Formatted at 2026-08-07 15:12:19 on dist-test-slave-42z9"
server_key: "60ebdbd2a775841b090c6ae3d4e7965a"
server_key_iv: "d4fce19af8a464b86c3052850ec46c1b"
server_key_version: "encryptionkey@0"
I20260807 15:12:19.591969 11834 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786115480851507-30472-0/minicluster-data/ts-1/wal/instance:
uuid: "2c70de56c780438c8ef2c1dffcde7d90"
format_stamp: "Formatted at 2026-08-07 15:12:19 on dist-test-slave-42z9"
server_key: "60ebdbd2a775841b090c6ae3d4e7965a"
server_key_iv: "d4fce19af8a464b86c3052850ec46c1b"
server_key_version: "encryptionkey@0"
I20260807 15:12:19.593577 11834 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.004s sys 0.000s
I20260807 15:12:19.594507 11854 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:12:19.594668 11834 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:12:19.594758 11834 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786115480851507-30472-0/minicluster-data/ts-1/data,/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786115480851507-30472-0/minicluster-data/ts-1/wal
uuid: "2c70de56c780438c8ef2c1dffcde7d90"
format_stamp: "Formatted at 2026-08-07 15:12:19 on dist-test-slave-42z9"
server_key: "60ebdbd2a775841b090c6ae3d4e7965a"
server_key_iv: "d4fce19af8a464b86c3052850ec46c1b"
server_key_version: "encryptionkey@0"
I20260807 15:12:19.594836 11834 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786115480851507-30472-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786115480851507-30472-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786115480851507-30472-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:12:19.604077 11834 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:12:19.604491 11834 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:12:19.604637 11834 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:12:19.604913 11834 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:12:19.605300 11834 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:12:19.605337 11834 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:12:19.605372 11834 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:12:19.605396 11834 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:12:19.612429 11834 rpc_server.cc:307] RPC server started. Bound to: 127.29.194.2:44911
I20260807 15:12:19.612473 12023 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.194.2:44911 every 8 connection(s)
I20260807 15:12:19.612820 11834 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786115480851507-30472-0/minicluster-data/ts-1/data/info.pb
I20260807 15:12:19.618671 11613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:19.613597 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.194.2:55695 (local address 127.29.194.62:41515)
0807 15:12:19.613645 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0807 15:12:19.613647 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:12:19.613940 (+ 293us) server_negotiation.cc:407] Connection header received
0807 15:12:19.614424 (+ 484us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:12:19.614425 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:12:19.614436 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:12:19.614493 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:12:19.615577 (+ 1084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:19.616095 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:19.617283 (+ 1188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:19.617519 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:19.617585 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:12:19.618435 (+ 850us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:12:19.618437 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:12:19.618437 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:12:19.618441 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:12:19.618455 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:12:19.618460 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:12:19.618505 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0807 15:12:19.618575 (+ 70us) server_negotiation.cc:299] Negotiation successful
0807 15:12:19.618603 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260807 15:12:19.618898 12026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:19.613605 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:41515 (local address 127.29.194.2:55695)
0807 15:12:19.613769 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0807 15:12:19.613804 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0807 15:12:19.614309 (+ 505us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:12:19.614760 (+ 451us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:12:19.614767 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:12:19.614781 (+ 14us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:12:19.615419 (+ 638us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:12:19.615425 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:12:19.616235 (+ 810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:12:19.616237 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:12:19.617153 (+ 916us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:12:19.617156 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:12:19.617635 (+ 479us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:12:19.617636 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:12:19.617767 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:12:19.618294 (+ 527us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 15:12:19.618294 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:12:19.618324 (+ 30us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 15:12:19.618350 (+ 26us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:12:19.618512 (+ 162us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:12:19.618516 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:12:19.618518 (+ 2us) client_negotiation.cc:769] Sending connection context
0807 15:12:19.618568 (+ 50us) client_negotiation.cc:240] Negotiation successful
0807 15:12:19.618659 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":68,"threads_started":1}
I20260807 15:12:19.619666 12024 heartbeater.cc:347] Connected to a master server at 127.29.194.62:41515
I20260807 15:12:19.619820 12024 heartbeater.cc:464] Registering TS with master...
I20260807 15:12:19.620074 12024 heartbeater.cc:511] Master 127.29.194.62:41515 requested a full tablet report, sending...
I20260807 15:12:19.620657 11547 ts_manager.cc:194] Registered new tserver with Master: 2c70de56c780438c8ef2c1dffcde7d90 (127.29.194.2:44911)
I20260807 15:12:19.621107 11547 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.29.194.2:55695
I20260807 15:12:19.622655 30472 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7fZSIL/build/release/bin/kudu as pid 11834
I20260807 15:12:19.622737 30472 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786115480851507-30472-0/minicluster-data/ts-1/wal/instance
I20260807 15:12:19.622897 30472 external_mini_cluster.cc:1428] Setting key 4ac1f1f88d5fae31232640c9fecdbc70
I20260807 15:12:19.624397 30472 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7fZSIL/build/release/bin/kudu
/tmp/dist-test-task7fZSIL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786115480851507-30472-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786115480851507-30472-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786115480851507-30472-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786115480851507-30472-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.194.3:0
--local_ip_for_outbound_sockets=127.29.194.3
--webserver_interface=127.29.194.3
--webserver_port=0
--tserver_master_addrs=127.29.194.62:41515
--builtin_ntp_servers=127.29.194.20:35849
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260807 15:12:19.713011 12027 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:12:19.713256 12027 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:12:19.713279 12027 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:12:19.714735 12027 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:12:19.714805 12027 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:12:19.714835 12027 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.194.3
I20260807 15:12:19.716327 12027 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.194.20:35849
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786115480851507-30472-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786115480851507-30472-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.194.3:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786115480851507-30472-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.194.3
--webserver_port=0
--tserver_master_addrs=127.29.194.62:41515
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12027
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.194.3
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786115480851507-30472-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:12:19.716564 12027 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:12:19.716853 12027 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:12:19.722499 12037 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:12:19.722563 12027 server_base.cc:1034] running on GCE node
W20260807 15:12:19.722509 12034 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:12:19.722499 12042 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:12:19.723276 12027 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:12:19.723618 12027 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:12:19.724829 12027 hybrid_clock.cc:648] HybridClock initialized: now 1786115539724772 us; error 73 us; skew 500 ppm
I20260807 15:12:19.726438 12027 webserver.cc:460] Webserver started at http://127.29.194.3:37867/ using document root <none> and password file <none>
I20260807 15:12:19.726693 12027 fs_manager.cc:359] Metadata directory not provided
I20260807 15:12:19.726745 12027 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:12:19.726874 12027 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:12:19.728008 12027 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786115480851507-30472-0/minicluster-data/ts-2/data/instance:
uuid: "0df5354f54f1459183a4c12993ce2c00"
format_stamp: "Formatted at 2026-08-07 15:12:19 on dist-test-slave-42z9"
server_key: "d405aa0c4fc11f57ec85515d96d6286b"
server_key_iv: "4b803730ffbcccba18a2fb4df4303413"
server_key_version: "encryptionkey@0"
I20260807 15:12:19.728351 12027 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786115480851507-30472-0/minicluster-data/ts-2/wal/instance:
uuid: "0df5354f54f1459183a4c12993ce2c00"
format_stamp: "Formatted at 2026-08-07 15:12:19 on dist-test-slave-42z9"
server_key: "d405aa0c4fc11f57ec85515d96d6286b"
server_key_iv: "4b803730ffbcccba18a2fb4df4303413"
server_key_version: "encryptionkey@0"
I20260807 15:12:19.730685 12027 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.004s
I20260807 15:12:19.731521 12050 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:12:19.731724 12027 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:12:19.731827 12027 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786115480851507-30472-0/minicluster-data/ts-2/data,/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786115480851507-30472-0/minicluster-data/ts-2/wal
uuid: "0df5354f54f1459183a4c12993ce2c00"
format_stamp: "Formatted at 2026-08-07 15:12:19 on dist-test-slave-42z9"
server_key: "d405aa0c4fc11f57ec85515d96d6286b"
server_key_iv: "4b803730ffbcccba18a2fb4df4303413"
server_key_version: "encryptionkey@0"
I20260807 15:12:19.731899 12027 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786115480851507-30472-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786115480851507-30472-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786115480851507-30472-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:12:19.759616 12027 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:12:19.760046 12027 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:12:19.760192 12027 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:12:19.760465 12027 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:12:19.760812 12027 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:12:19.760847 12027 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:12:19.760878 12027 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:12:19.760900 12027 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:12:19.767593 12027 rpc_server.cc:307] RPC server started. Bound to: 127.29.194.3:40241
I20260807 15:12:19.767760 12224 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.194.3:40241 every 8 connection(s)
I20260807 15:12:19.768049 12027 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786115480851507-30472-0/minicluster-data/ts-2/data/info.pb
I20260807 15:12:19.770855 30472 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7fZSIL/build/release/bin/kudu as pid 12027
I20260807 15:12:19.770941 30472 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786115480851507-30472-0/minicluster-data/ts-2/wal/instance
I20260807 15:12:19.771116 30472 external_mini_cluster.cc:1428] Setting key fe2f802665eb357dc6af7b77bcfc0241
I20260807 15:12:19.774245 11613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:19.768909 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.194.3:36073 (local address 127.29.194.62:41515)
0807 15:12:19.768965 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0807 15:12:19.768968 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:12:19.769246 (+ 278us) server_negotiation.cc:407] Connection header received
0807 15:12:19.769787 (+ 541us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:12:19.769788 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:12:19.769800 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:12:19.769868 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:12:19.770642 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:19.771265 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:19.772547 (+ 1282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:19.772743 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:19.772803 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:12:19.773857 (+ 1054us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:12:19.773860 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:12:19.773860 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:12:19.773868 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:12:19.773906 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:12:19.773912 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:12:19.774008 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0807 15:12:19.774144 (+ 136us) server_negotiation.cc:299] Negotiation successful
0807 15:12:19.774178 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260807 15:12:19.774366 12230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:19.768848 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:41515 (local address 127.29.194.3:36073)
0807 15:12:19.769060 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0807 15:12:19.769101 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0807 15:12:19.769641 (+ 540us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:12:19.769981 (+ 340us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:12:19.769986 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:12:19.769995 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:12:19.770533 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:12:19.770537 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:12:19.771383 (+ 846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:12:19.771384 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:12:19.772468 (+ 1084us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:12:19.772472 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:12:19.772866 (+ 394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:12:19.772867 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:12:19.773008 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:12:19.773635 (+ 627us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 15:12:19.773635 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:12:19.773666 (+ 31us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 15:12:19.773693 (+ 27us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:12:19.774016 (+ 323us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:12:19.774020 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:12:19.774023 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 15:12:19.774082 (+ 59us) client_negotiation.cc:240] Negotiation successful
0807 15:12:19.774178 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":85,"threads_started":1}
I20260807 15:12:19.774796 12226 heartbeater.cc:347] Connected to a master server at 127.29.194.62:41515
I20260807 15:12:19.774892 12226 heartbeater.cc:464] Registering TS with master...
I20260807 15:12:19.775080 12226 heartbeater.cc:511] Master 127.29.194.62:41515 requested a full tablet report, sending...
I20260807 15:12:19.775431 11547 ts_manager.cc:194] Registered new tserver with Master: 0df5354f54f1459183a4c12993ce2c00 (127.29.194.3:40241)
I20260807 15:12:19.775808 11547 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.29.194.3:36073
I20260807 15:12:19.783635 30472 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 15:12:19.785547 30472 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7fZSIL/build/release/bin/kudu with pid 11644
W20260807 15:12:19.785552 11613 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:12:19.785015 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45648 (local address 127.29.194.62:41515)
0807 15:12:19.785064 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0807 15:12:19.785066 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:12:19.785162 (+ 96us) server_negotiation.cc:407] Connection header received
0807 15:12:19.785209 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:12:19.785210 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:12:19.785222 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:12:19.785283 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:12:19.785479 (+ 196us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45648: BlockingRecv error: recv got EOF from 127.0.0.1:45648 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
I20260807 15:12:19.801235 30472 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7fZSIL/build/release/bin/kudu with pid 11834
I20260807 15:12:19.813308 30472 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7fZSIL/build/release/bin/kudu with pid 12027
I20260807 15:12:19.842564 30472 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7fZSIL/build/release/bin/kudu with pid 11501
2026-08-07T15:12:19Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (716 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-07T15:12:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T15:12:19Z Disabled control of system clock
I20260807 15:12:19.878954 30472 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7fZSIL/build/release/bin/kudu
/tmp/dist-test-task7fZSIL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786115480851507-30472-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786115480851507-30472-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786115480851507-30472-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786115480851507-30472-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.194.62:37753
--webserver_interface=127.29.194.62
--webserver_port=0
--builtin_ntp_servers=127.29.194.20:40735
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.194.62:37753
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260807 15:12:19.969007 12241 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:12:19.969257 12241 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:12:19.969281 12241 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:12:19.971089 12241 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:12:19.971182 12241 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:12:19.971194 12241 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:12:19.971207 12241 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:12:19.971218 12241 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:12:19.972762 12241 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.194.20:40735
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786115480851507-30472-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786115480851507-30472-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.194.62:37753
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.194.62:37753
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786115480851507-30472-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.194.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12241
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786115480851507-30472-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:12:19.973021 12241 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:12:19.973290 12241 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:12:19.978619 12252 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:12:19.978619 12248 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:12:19.978648 12247 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:12:19.978883 12241 server_base.cc:1034] running on GCE node
I20260807 15:12:19.979250 12241 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:12:19.979573 12241 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:12:19.980737 12241 hybrid_clock.cc:648] HybridClock initialized: now 1786115539980724 us; error 32 us; skew 500 ppm
I20260807 15:12:19.982221 12241 webserver.cc:460] Webserver started at http://127.29.194.62:39905/ using document root <none> and password file <none>
I20260807 15:12:19.982460 12241 fs_manager.cc:359] Metadata directory not provided
I20260807 15:12:19.982512 12241 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:12:19.982659 12241 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:12:19.983789 12241 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786115480851507-30472-0/minicluster-data/master-0/data/instance:
uuid: "5e318c6f2f2444bdb6ebf5b0fc0f032e"
format_stamp: "Formatted at 2026-08-07 15:12:19 on dist-test-slave-42z9"
server_key: "67bd9417b25636693b21602a4ff42da8"
server_key_iv: "25a8313eab2eab2b9f94ee9d52a10227"
server_key_version: "encryptionkey@0"
I20260807 15:12:19.984141 12241 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786115480851507-30472-0/minicluster-data/master-0/wal/instance:
uuid: "5e318c6f2f2444bdb6ebf5b0fc0f032e"
format_stamp: "Formatted at 2026-08-07 15:12:19 on dist-test-slave-42z9"
server_key: "67bd9417b25636693b21602a4ff42da8"
server_key_iv: "25a8313eab2eab2b9f94ee9d52a10227"
server_key_version: "encryptionkey@0"
I20260807 15:12:19.986323 12241 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.004s sys 0.000s
I20260807 15:12:19.987540 12262 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:12:19.987799 12241 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:12:19.987910 12241 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786115480851507-30472-0/minicluster-data/master-0/data,/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786115480851507-30472-0/minicluster-data/master-0/wal
uuid: "5e318c6f2f2444bdb6ebf5b0fc0f032e"
format_stamp: "Formatted at 2026-08-07 15:12:19 on dist-test-slave-42z9"
server_key: "67bd9417b25636693b21602a4ff42da8"
server_key_iv: "25a8313eab2eab2b9f94ee9d52a10227"
server_key_version: "encryptionkey@0"
I20260807 15:12:19.988023 12241 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786115480851507-30472-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786115480851507-30472-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786115480851507-30472-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:12:19.991618 12241 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:12:19.991906 12241 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:12:19.992028 12241 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:12:19.996203 12241 rpc_server.cc:307] RPC server started. Bound to: 127.29.194.62:37753
I20260807 15:12:19.996376 12350 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.194.62:37753 every 8 connection(s)
I20260807 15:12:19.996656 12241 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786115480851507-30472-0/minicluster-data/master-0/data/info.pb
I20260807 15:12:19.997419 12351 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:12:20.000422 12351 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5e318c6f2f2444bdb6ebf5b0fc0f032e: Bootstrap starting.
I20260807 15:12:20.001068 12351 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5e318c6f2f2444bdb6ebf5b0fc0f032e: Neither blocks nor log segments found. Creating new log.
I20260807 15:12:20.001381 12351 log.cc:824] T 00000000000000000000000000000000 P 5e318c6f2f2444bdb6ebf5b0fc0f032e: Log is configured to *not* fsync() on all Append() calls
I20260807 15:12:20.002420 12351 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5e318c6f2f2444bdb6ebf5b0fc0f032e: No bootstrap required, opened a new log
I20260807 15:12:20.004144 12351 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5e318c6f2f2444bdb6ebf5b0fc0f032e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e318c6f2f2444bdb6ebf5b0fc0f032e" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 37753 } }
I20260807 15:12:20.004304 12351 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5e318c6f2f2444bdb6ebf5b0fc0f032e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:12:20.004326 12351 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5e318c6f2f2444bdb6ebf5b0fc0f032e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e318c6f2f2444bdb6ebf5b0fc0f032e, State: Initialized, Role: FOLLOWER
I20260807 15:12:20.004447 12351 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5e318c6f2f2444bdb6ebf5b0fc0f032e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e318c6f2f2444bdb6ebf5b0fc0f032e" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 37753 } }
I20260807 15:12:20.004539 12351 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5e318c6f2f2444bdb6ebf5b0fc0f032e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:12:20.004575 12351 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5e318c6f2f2444bdb6ebf5b0fc0f032e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:12:20.004612 12351 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5e318c6f2f2444bdb6ebf5b0fc0f032e [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:12:20.005215 12351 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5e318c6f2f2444bdb6ebf5b0fc0f032e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e318c6f2f2444bdb6ebf5b0fc0f032e" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 37753 } }
I20260807 15:12:20.005313 12351 leader_election.cc:302] T 00000000000000000000000000000000 P 5e318c6f2f2444bdb6ebf5b0fc0f032e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5e318c6f2f2444bdb6ebf5b0fc0f032e; no voters:
I20260807 15:12:20.005487 12351 leader_election.cc:288] T 00000000000000000000000000000000 P 5e318c6f2f2444bdb6ebf5b0fc0f032e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 15:12:20.005558 12358 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5e318c6f2f2444bdb6ebf5b0fc0f032e [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:12:20.005676 30472 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7fZSIL/build/release/bin/kudu as pid 12241
I20260807 15:12:20.005730 12358 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5e318c6f2f2444bdb6ebf5b0fc0f032e [term 1 LEADER]: Becoming Leader. State: Replica: 5e318c6f2f2444bdb6ebf5b0fc0f032e, State: Running, Role: LEADER
I20260807 15:12:20.005767 30472 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786115480851507-30472-0/minicluster-data/master-0/wal/instance
I20260807 15:12:20.005815 12351 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5e318c6f2f2444bdb6ebf5b0fc0f032e [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:12:20.005821 12358 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5e318c6f2f2444bdb6ebf5b0fc0f032e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e318c6f2f2444bdb6ebf5b0fc0f032e" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 37753 } }
I20260807 15:12:20.005966 30472 external_mini_cluster.cc:1428] Setting key 4d97be3d987c1c43110b4a0065de0782
I20260807 15:12:20.006239 12359 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5e318c6f2f2444bdb6ebf5b0fc0f032e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5e318c6f2f2444bdb6ebf5b0fc0f032e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e318c6f2f2444bdb6ebf5b0fc0f032e" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 37753 } } }
I20260807 15:12:20.006318 12359 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5e318c6f2f2444bdb6ebf5b0fc0f032e [sys.catalog]: This master's current role is: LEADER
I20260807 15:12:20.006494 12361 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5e318c6f2f2444bdb6ebf5b0fc0f032e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5e318c6f2f2444bdb6ebf5b0fc0f032e. Latest consensus state: current_term: 1 leader_uuid: "5e318c6f2f2444bdb6ebf5b0fc0f032e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e318c6f2f2444bdb6ebf5b0fc0f032e" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 37753 } } }
I20260807 15:12:20.006547 12361 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5e318c6f2f2444bdb6ebf5b0fc0f032e [sys.catalog]: This master's current role is: LEADER
I20260807 15:12:20.006672 12367 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:12:20.007905 12367 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:12:20.010113 12373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:20.006737 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40418 (local address 127.29.194.62:37753)
0807 15:12:20.007193 (+ 456us) server_negotiation.cc:206] Beginning negotiation
0807 15:12:20.007202 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 15:12:20.007236 (+ 34us) server_negotiation.cc:407] Connection header received
0807 15:12:20.007549 (+ 313us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:12:20.007561 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:12:20.007632 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:12:20.007936 (+ 304us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:12:20.009668 (+ 1732us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:12:20.009672 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:12:20.009672 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:12:20.009677 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:12:20.009717 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:12:20.009732 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:12:20.009797 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0807 15:12:20.009861 (+ 64us) server_negotiation.cc:299] Negotiation successful
0807 15:12:20.009924 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":377,"thread_start_us":64,"threads_started":1}
I20260807 15:12:20.010170 12367 catalog_manager.cc:1347] Generated new cluster ID: cbc8cefa08404aa488a33e0bdec0c821
I20260807 15:12:20.010205 12367 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:12:20.019981 12367 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:12:20.020171 12367 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:12:20.024313 12367 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5e318c6f2f2444bdb6ebf5b0fc0f032e: Generated new TSK 0
I20260807 15:12:20.024544 12367 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 15:12:20.031919 30472 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7fZSIL/build/release/bin/kudu
/tmp/dist-test-task7fZSIL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786115480851507-30472-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786115480851507-30472-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786115480851507-30472-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786115480851507-30472-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.194.1:0
--local_ip_for_outbound_sockets=127.29.194.1
--webserver_interface=127.29.194.1
--webserver_port=0
--tserver_master_addrs=127.29.194.62:37753
--builtin_ntp_servers=127.29.194.20:40735
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260807 15:12:20.126338 12387 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:12:20.126567 12387 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:12:20.126602 12387 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:12:20.128448 12387 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:12:20.128553 12387 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:12:20.128605 12387 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.194.1
I20260807 15:12:20.130395 12387 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.194.20:40735
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786115480851507-30472-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786115480851507-30472-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.194.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786115480851507-30472-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.194.1
--webserver_port=0
--tserver_master_addrs=127.29.194.62:37753
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12387
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.194.1
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786115480851507-30472-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:12:20.130700 12387 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:12:20.131033 12387 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:12:20.136574 12399 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:12:20.136556 12393 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:12:20.136555 12394 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:12:20.137169 12387 server_base.cc:1034] running on GCE node
I20260807 15:12:20.137552 12387 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:12:20.137885 12387 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:12:20.139091 12387 hybrid_clock.cc:648] HybridClock initialized: now 1786115540139059 us; error 46 us; skew 500 ppm
I20260807 15:12:20.140842 12387 webserver.cc:460] Webserver started at http://127.29.194.1:40443/ using document root <none> and password file <none>
I20260807 15:12:20.141171 12387 fs_manager.cc:359] Metadata directory not provided
I20260807 15:12:20.141232 12387 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:12:20.141580 12387 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:12:20.142727 12387 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786115480851507-30472-0/minicluster-data/ts-0/data/instance:
uuid: "aea62624408a4aebaa587eda0d34f429"
format_stamp: "Formatted at 2026-08-07 15:12:20 on dist-test-slave-42z9"
server_key: "c2542a9820bfc44ee3a3e28543641d09"
server_key_iv: "0d2418b581730932481dcac35ff40402"
server_key_version: "encryptionkey@0"
I20260807 15:12:20.143086 12387 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786115480851507-30472-0/minicluster-data/ts-0/wal/instance:
uuid: "aea62624408a4aebaa587eda0d34f429"
format_stamp: "Formatted at 2026-08-07 15:12:20 on dist-test-slave-42z9"
server_key: "c2542a9820bfc44ee3a3e28543641d09"
server_key_iv: "0d2418b581730932481dcac35ff40402"
server_key_version: "encryptionkey@0"
I20260807 15:12:20.144558 12387 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 15:12:20.145613 12407 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:12:20.145823 12387 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:12:20.146039 12387 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786115480851507-30472-0/minicluster-data/ts-0/data,/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786115480851507-30472-0/minicluster-data/ts-0/wal
uuid: "aea62624408a4aebaa587eda0d34f429"
format_stamp: "Formatted at 2026-08-07 15:12:20 on dist-test-slave-42z9"
server_key: "c2542a9820bfc44ee3a3e28543641d09"
server_key_iv: "0d2418b581730932481dcac35ff40402"
server_key_version: "encryptionkey@0"
I20260807 15:12:20.146121 12387 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786115480851507-30472-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786115480851507-30472-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786115480851507-30472-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:12:20.149596 12387 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:12:20.149874 12387 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:12:20.150002 12387 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:12:20.150211 12387 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:12:20.150503 12387 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:12:20.150533 12387 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:12:20.150552 12387 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:12:20.150563 12387 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:12:20.156504 12387 rpc_server.cc:307] RPC server started. Bound to: 127.29.194.1:36305
I20260807 15:12:20.156690 12577 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.194.1:36305 every 8 connection(s)
I20260807 15:12:20.156956 12387 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786115480851507-30472-0/minicluster-data/ts-0/data/info.pb
I20260807 15:12:20.160138 12373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:20.157756 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.194.1:55203 (local address 127.29.194.62:37753)
0807 15:12:20.157810 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0807 15:12:20.157813 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:12:20.158093 (+ 280us) server_negotiation.cc:407] Connection header received
0807 15:12:20.158566 (+ 473us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:12:20.158567 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:12:20.158573 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:12:20.158630 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:12:20.159815 (+ 1185us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:12:20.159817 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:12:20.159818 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:12:20.159822 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:12:20.159847 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:12:20.159852 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:12:20.159912 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0807 15:12:20.160063 (+ 151us) server_negotiation.cc:299] Negotiation successful
0807 15:12:20.160077 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260807 15:12:20.160303 12582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:20.157745 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:37753 (local address 127.29.194.1:55203)
0807 15:12:20.157953 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0807 15:12:20.157989 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0807 15:12:20.158463 (+ 474us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:12:20.158694 (+ 231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:12:20.158697 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:12:20.158706 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:12:20.159649 (+ 943us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 15:12:20.159650 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:12:20.159681 (+ 31us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 15:12:20.159712 (+ 31us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:12:20.159943 (+ 231us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:12:20.159951 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:12:20.159954 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 15:12:20.160039 (+ 85us) client_negotiation.cc:240] Negotiation successful
0807 15:12:20.160096 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":104,"threads_started":1}
I20260807 15:12:20.160874 12578 heartbeater.cc:347] Connected to a master server at 127.29.194.62:37753
I20260807 15:12:20.161042 12578 heartbeater.cc:464] Registering TS with master...
I20260807 15:12:20.161232 30472 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7fZSIL/build/release/bin/kudu as pid 12387
I20260807 15:12:20.161273 12578 heartbeater.cc:511] Master 127.29.194.62:37753 requested a full tablet report, sending...
I20260807 15:12:20.161336 30472 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786115480851507-30472-0/minicluster-data/ts-0/wal/instance
I20260807 15:12:20.161592 30472 external_mini_cluster.cc:1428] Setting key e87e00b20a95ee64c989c8af694e3723
I20260807 15:12:20.162199 12290 ts_manager.cc:194] Registered new tserver with Master: aea62624408a4aebaa587eda0d34f429 (127.29.194.1:36305)
I20260807 15:12:20.162935 30472 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7fZSIL/build/release/bin/kudu
/tmp/dist-test-task7fZSIL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786115480851507-30472-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786115480851507-30472-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786115480851507-30472-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786115480851507-30472-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.194.2:0
--local_ip_for_outbound_sockets=127.29.194.2
--webserver_interface=127.29.194.2
--webserver_port=0
--tserver_master_addrs=127.29.194.62:37753
--builtin_ntp_servers=127.29.194.20:40735
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260807 15:12:20.248061 12583 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:12:20.248301 12583 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:12:20.248325 12583 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:12:20.249679 12583 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:12:20.249734 12583 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:12:20.249764 12583 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.194.2
I20260807 15:12:20.251240 12583 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.194.20:40735
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786115480851507-30472-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786115480851507-30472-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.194.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786115480851507-30472-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.194.2
--webserver_port=0
--tserver_master_addrs=127.29.194.62:37753
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12583
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.194.2
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786115480851507-30472-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:12:20.251487 12583 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:12:20.251762 12583 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:12:20.257776 12599 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:12:20.257784 12596 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:12:20.258068 12595 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:12:20.257999 12583 server_base.cc:1034] running on GCE node
I20260807 15:12:20.258467 12583 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:12:20.258759 12583 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:12:20.259933 12583 hybrid_clock.cc:648] HybridClock initialized: now 1786115540259879 us; error 73 us; skew 500 ppm
I20260807 15:12:20.261575 12583 webserver.cc:460] Webserver started at http://127.29.194.2:42487/ using document root <none> and password file <none>
I20260807 15:12:20.262233 12583 fs_manager.cc:359] Metadata directory not provided
I20260807 15:12:20.262328 12583 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:12:20.262527 12583 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:12:20.264135 12583 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786115480851507-30472-0/minicluster-data/ts-1/data/instance:
uuid: "1503522c2df7416cbea1a546e9fd4a8b"
format_stamp: "Formatted at 2026-08-07 15:12:20 on dist-test-slave-42z9"
server_key: "52f97a8b78da98e502d8c2fe54bd3095"
server_key_iv: "50d463079b55ab01d46f8a40c30b9b61"
server_key_version: "encryptionkey@0"
I20260807 15:12:20.264618 12583 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786115480851507-30472-0/minicluster-data/ts-1/wal/instance:
uuid: "1503522c2df7416cbea1a546e9fd4a8b"
format_stamp: "Formatted at 2026-08-07 15:12:20 on dist-test-slave-42z9"
server_key: "52f97a8b78da98e502d8c2fe54bd3095"
server_key_iv: "50d463079b55ab01d46f8a40c30b9b61"
server_key_version: "encryptionkey@0"
I20260807 15:12:20.266389 12583 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.004s sys 0.000s
I20260807 15:12:20.267693 12606 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:12:20.268014 12583 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:12:20.268126 12583 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786115480851507-30472-0/minicluster-data/ts-1/data,/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786115480851507-30472-0/minicluster-data/ts-1/wal
uuid: "1503522c2df7416cbea1a546e9fd4a8b"
format_stamp: "Formatted at 2026-08-07 15:12:20 on dist-test-slave-42z9"
server_key: "52f97a8b78da98e502d8c2fe54bd3095"
server_key_iv: "50d463079b55ab01d46f8a40c30b9b61"
server_key_version: "encryptionkey@0"
I20260807 15:12:20.268209 12583 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786115480851507-30472-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786115480851507-30472-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786115480851507-30472-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:12:20.272644 12583 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:12:20.272948 12583 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:12:20.273087 12583 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:12:20.273296 12583 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:12:20.273680 12583 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:12:20.273726 12583 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:12:20.273756 12583 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:12:20.273779 12583 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:12:20.284181 12583 rpc_server.cc:307] RPC server started. Bound to: 127.29.194.2:43389
I20260807 15:12:20.284315 12777 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.194.2:43389 every 8 connection(s)
I20260807 15:12:20.284597 12583 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786115480851507-30472-0/minicluster-data/ts-1/data/info.pb
I20260807 15:12:20.288273 12373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:20.285539 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.194.2:60729 (local address 127.29.194.62:37753)
0807 15:12:20.285636 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0807 15:12:20.285639 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:12:20.285866 (+ 227us) server_negotiation.cc:407] Connection header received
0807 15:12:20.286515 (+ 649us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:12:20.286516 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:12:20.286525 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:12:20.286583 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:12:20.287971 (+ 1388us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:12:20.287974 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:12:20.287974 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:12:20.287979 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:12:20.288002 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:12:20.288007 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:12:20.288058 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0807 15:12:20.288142 (+ 84us) server_negotiation.cc:299] Negotiation successful
0807 15:12:20.288162 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260807 15:12:20.288501 12781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:20.285337 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:37753 (local address 127.29.194.2:60729)
0807 15:12:20.285663 (+ 326us) negotiation.cc:107] Waiting for socket to connect
0807 15:12:20.285735 (+ 72us) client_negotiation.cc:174] Beginning negotiation
0807 15:12:20.286397 (+ 662us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:12:20.286676 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:12:20.286682 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:12:20.286695 (+ 13us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:12:20.287788 (+ 1093us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 15:12:20.287789 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:12:20.287828 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 15:12:20.287868 (+ 40us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:12:20.288062 (+ 194us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:12:20.288068 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:12:20.288073 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 15:12:20.288152 (+ 79us) client_negotiation.cc:240] Negotiation successful
0807 15:12:20.288232 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":79,"threads_started":1}
I20260807 15:12:20.289278 12779 heartbeater.cc:347] Connected to a master server at 127.29.194.62:37753
I20260807 15:12:20.289486 12779 heartbeater.cc:464] Registering TS with master...
I20260807 15:12:20.289661 12779 heartbeater.cc:511] Master 127.29.194.62:37753 requested a full tablet report, sending...
I20260807 15:12:20.290246 12290 ts_manager.cc:194] Registered new tserver with Master: 1503522c2df7416cbea1a546e9fd4a8b (127.29.194.2:43389)
I20260807 15:12:20.290855 30472 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7fZSIL/build/release/bin/kudu as pid 12583
I20260807 15:12:20.290937 30472 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786115480851507-30472-0/minicluster-data/ts-1/wal/instance
I20260807 15:12:20.291145 30472 external_mini_cluster.cc:1428] Setting key 78d350a152f0b2cf28f2e8d47e971abf
I20260807 15:12:20.292411 30472 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7fZSIL/build/release/bin/kudu
/tmp/dist-test-task7fZSIL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786115480851507-30472-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786115480851507-30472-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786115480851507-30472-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786115480851507-30472-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.194.3:0
--local_ip_for_outbound_sockets=127.29.194.3
--webserver_interface=127.29.194.3
--webserver_port=0
--tserver_master_addrs=127.29.194.62:37753
--builtin_ntp_servers=127.29.194.20:40735
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260807 15:12:20.383498 12782 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:12:20.383741 12782 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:12:20.383774 12782 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:12:20.385079 12782 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:12:20.385131 12782 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:12:20.385160 12782 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.194.3
I20260807 15:12:20.386723 12782 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.194.20:40735
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786115480851507-30472-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786115480851507-30472-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.194.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786115480851507-30472-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.194.3
--webserver_port=0
--tserver_master_addrs=127.29.194.62:37753
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12782
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.194.3
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786115480851507-30472-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:12:20.386962 12782 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:12:20.387243 12782 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:12:20.392722 12791 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:12:20.392699 12794 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:12:20.392956 12782 server_base.cc:1034] running on GCE node
W20260807 15:12:20.392705 12790 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:12:20.393407 12782 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:12:20.393805 12782 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:12:20.394994 12782 hybrid_clock.cc:648] HybridClock initialized: now 1786115540394974 us; error 42 us; skew 500 ppm
I20260807 15:12:20.396445 12782 webserver.cc:460] Webserver started at http://127.29.194.3:35109/ using document root <none> and password file <none>
I20260807 15:12:20.396711 12782 fs_manager.cc:359] Metadata directory not provided
I20260807 15:12:20.396767 12782 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:12:20.396901 12782 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:12:20.398072 12782 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786115480851507-30472-0/minicluster-data/ts-2/data/instance:
uuid: "c0b158c7d080491b8cda971754fab3cf"
format_stamp: "Formatted at 2026-08-07 15:12:20 on dist-test-slave-42z9"
server_key: "27ef3082b905f784a136f4081bf0af1c"
server_key_iv: "2aa077e95d9f40a79a1769b7be1f6af8"
server_key_version: "encryptionkey@0"
I20260807 15:12:20.398443 12782 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786115480851507-30472-0/minicluster-data/ts-2/wal/instance:
uuid: "c0b158c7d080491b8cda971754fab3cf"
format_stamp: "Formatted at 2026-08-07 15:12:20 on dist-test-slave-42z9"
server_key: "27ef3082b905f784a136f4081bf0af1c"
server_key_iv: "2aa077e95d9f40a79a1769b7be1f6af8"
server_key_version: "encryptionkey@0"
I20260807 15:12:20.399847 12782 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 15:12:20.400792 12802 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:12:20.401091 12782 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:12:20.401214 12782 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786115480851507-30472-0/minicluster-data/ts-2/data,/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786115480851507-30472-0/minicluster-data/ts-2/wal
uuid: "c0b158c7d080491b8cda971754fab3cf"
format_stamp: "Formatted at 2026-08-07 15:12:20 on dist-test-slave-42z9"
server_key: "27ef3082b905f784a136f4081bf0af1c"
server_key_iv: "2aa077e95d9f40a79a1769b7be1f6af8"
server_key_version: "encryptionkey@0"
I20260807 15:12:20.401315 12782 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786115480851507-30472-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786115480851507-30472-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786115480851507-30472-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:12:20.405129 12782 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:12:20.405391 12782 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:12:20.405542 12782 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:12:20.405721 12782 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:12:20.406111 12782 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:12:20.406150 12782 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:12:20.406173 12782 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:12:20.406184 12782 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:12:20.413569 12782 rpc_server.cc:307] RPC server started. Bound to: 127.29.194.3:41259
I20260807 15:12:20.413623 12974 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.194.3:41259 every 8 connection(s)
I20260807 15:12:20.414108 12782 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786115480851507-30472-0/minicluster-data/ts-2/data/info.pb
I20260807 15:12:20.418161 12373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:20.415165 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.194.3:50589 (local address 127.29.194.62:37753)
0807 15:12:20.415263 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0807 15:12:20.415266 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:12:20.415544 (+ 278us) server_negotiation.cc:407] Connection header received
0807 15:12:20.416099 (+ 555us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:12:20.416101 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:12:20.416117 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:12:20.416189 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:12:20.417921 (+ 1732us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:12:20.417922 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:12:20.417923 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:12:20.417928 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:12:20.417946 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:12:20.417952 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:12:20.418012 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0807 15:12:20.418079 (+ 67us) server_negotiation.cc:299] Negotiation successful
0807 15:12:20.418094 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260807 15:12:20.418335 12980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:20.415110 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:37753 (local address 127.29.194.3:50589)
0807 15:12:20.415373 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0807 15:12:20.415417 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0807 15:12:20.415935 (+ 518us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:12:20.416583 (+ 648us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:12:20.416587 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:12:20.416599 (+ 12us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:12:20.417704 (+ 1105us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 15:12:20.417705 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:12:20.417740 (+ 35us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 15:12:20.417773 (+ 33us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:12:20.418015 (+ 242us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:12:20.418018 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:12:20.418021 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 15:12:20.418079 (+ 58us) client_negotiation.cc:240] Negotiation successful
0807 15:12:20.418136 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":114,"threads_started":1}
I20260807 15:12:20.419000 12975 heartbeater.cc:347] Connected to a master server at 127.29.194.62:37753
I20260807 15:12:20.419106 12975 heartbeater.cc:464] Registering TS with master...
I20260807 15:12:20.419138 30472 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7fZSIL/build/release/bin/kudu as pid 12782
I20260807 15:12:20.419236 30472 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786115480851507-30472-0/minicluster-data/ts-2/wal/instance
I20260807 15:12:20.419287 12975 heartbeater.cc:511] Master 127.29.194.62:37753 requested a full tablet report, sending...
I20260807 15:12:20.419669 30472 external_mini_cluster.cc:1428] Setting key 0dc51aa8932fddae8b1cde2231da8536
I20260807 15:12:20.419872 12290 ts_manager.cc:194] Registered new tserver with Master: c0b158c7d080491b8cda971754fab3cf (127.29.194.3:41259)
I20260807 15:12:20.421044 30472 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260807 15:12:20.423254 12373 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:12:20.422595 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40432 (local address 127.29.194.62:37753)
0807 15:12:20.422654 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0807 15:12:20.422656 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:12:20.422738 (+ 82us) server_negotiation.cc:407] Connection header received
0807 15:12:20.422797 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:12:20.422798 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:12:20.422805 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:12:20.422855 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:12:20.423156 (+ 301us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40432: BlockingRecv error: recv got EOF from 127.0.0.1:40432 (error 108)
Metrics: {"server-negotiator.queue_time_us":29}
I20260807 15:12:20.423404 30472 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7fZSIL/build/release/bin/kudu with pid 12387
I20260807 15:12:20.435760 30472 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7fZSIL/build/release/bin/kudu with pid 12583
I20260807 15:12:20.461019 30472 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7fZSIL/build/release/bin/kudu with pid 12782
I20260807 15:12:20.476467 30472 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7fZSIL/build/release/bin/kudu with pid 12241
2026-08-07T15:12:20Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (633 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115480851507-30472-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 15:12:20 dist-test-slave-42z9 krb5kdc[12990](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 15:12:20 dist-test-slave-42z9 krb5kdc[12990](info): set up 2 sockets
Aug 07 15:12:20 dist-test-slave-42z9 krb5kdc[12990](info): commencing operation
krb5kdc: starting...
W20260807 15:12:21.634948 30472 mini_kdc.cc:121] Time spent starting KDC: real 1.131s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115480851507-30472-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115480851507-30472-0/krb5kdc/test-user.keytab.
Aug 07 15:12:21 dist-test-slave-42z9 krb5kdc[12990](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115541, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T15:12:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T15:12:21Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.194.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.194.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.194.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.62.keytab.
Entry for principal kudu/127.29.194.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.62.keytab.
I20260807 15:12:21.772564 30472 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7fZSIL/build/release/bin/kudu
/tmp/dist-test-task7fZSIL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115480851507-30472-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115480851507-30472-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115480851507-30472-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115480851507-30472-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.194.62:41747
--webserver_interface=127.29.194.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.62.keytab
--principal=kudu/127.29.194.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.194.20:39625
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.194.62:41747
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115480851507-30472-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115480851507-30472-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115480851507-30472-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:12:21.858687 13008 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:12:21.858950 13008 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:12:21.858986 13008 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:12:21.860268 13008 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:12:21.860328 13008 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:12:21.860347 13008 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:12:21.860364 13008 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:12:21.860379 13008 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:12:21.862087 13008 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.194.20:39625
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115480851507-30472-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115480851507-30472-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.194.62:41747
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.62.keytab
--principal=kudu/127.29.194.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.194.62:41747
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115480851507-30472-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.194.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13008
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115480851507-30472-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:12:21.862389 13008 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:12:21.862628 13008 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:12:21.868196 13019 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:12:21.868197 13017 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:12:21.868469 13008 server_base.cc:1034] running on GCE node
W20260807 15:12:21.868197 13016 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:12:21.868945 13008 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:12:21.869359 13008 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:12:21.870568 13008 hybrid_clock.cc:648] HybridClock initialized: now 1786115541870543 us; error 40 us; skew 500 ppm
Aug 07 15:12:21 dist-test-slave-42z9 krb5kdc[12990](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115541, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.194.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:12:21.873281 13008 init.cc:377] Logged in from keytab as kudu/127.29.194.62@KRBTEST.COM (short username kudu)
I20260807 15:12:21.873817 13008 webserver.cc:460] Webserver started at http://127.29.194.62:37683/ using document root <none> and password file <none>
I20260807 15:12:21.874027 13008 fs_manager.cc:359] Metadata directory not provided
I20260807 15:12:21.874078 13008 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:12:21.874183 13008 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:12:21.875243 13008 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115480851507-30472-0/minicluster-data/master-0/data/instance:
uuid: "d38713e0a7834cb39624cfd1d23d5ecb"
format_stamp: "Formatted at 2026-08-07 15:12:21 on dist-test-slave-42z9"
server_key: "18122356081b14d4dcb90812702cfd74"
server_key_iv: "58d3f59b0f5d1e8c8b776496e84730f0"
server_key_version: "encryptionkey@0"
I20260807 15:12:21.875559 13008 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115480851507-30472-0/minicluster-data/master-0/wal/instance:
uuid: "d38713e0a7834cb39624cfd1d23d5ecb"
format_stamp: "Formatted at 2026-08-07 15:12:21 on dist-test-slave-42z9"
server_key: "18122356081b14d4dcb90812702cfd74"
server_key_iv: "58d3f59b0f5d1e8c8b776496e84730f0"
server_key_version: "encryptionkey@0"
I20260807 15:12:21.877154 13008 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260807 15:12:21.878309 13033 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:12:21.878512 13008 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:12:21.878619 13008 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115480851507-30472-0/minicluster-data/master-0/data,/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115480851507-30472-0/minicluster-data/master-0/wal
uuid: "d38713e0a7834cb39624cfd1d23d5ecb"
format_stamp: "Formatted at 2026-08-07 15:12:21 on dist-test-slave-42z9"
server_key: "18122356081b14d4dcb90812702cfd74"
server_key_iv: "58d3f59b0f5d1e8c8b776496e84730f0"
server_key_version: "encryptionkey@0"
I20260807 15:12:21.878696 13008 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115480851507-30472-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115480851507-30472-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115480851507-30472-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:12:21.890611 13008 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:12:21.893057 13008 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:12:21.893182 13008 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:12:21.897773 13008 rpc_server.cc:307] RPC server started. Bound to: 127.29.194.62:41747
I20260807 15:12:21.897814 13112 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.194.62:41747 every 8 connection(s)
I20260807 15:12:21.898177 13008 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115480851507-30472-0/minicluster-data/master-0/data/info.pb
I20260807 15:12:21.899257 30472 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7fZSIL/build/release/bin/kudu as pid 13008
I20260807 15:12:21.899288 13114 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:12:21.899387 30472 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115480851507-30472-0/minicluster-data/master-0/wal/instance
I20260807 15:12:21.899613 30472 external_mini_cluster.cc:1428] Setting key 3238097c22313efef69322385a06d75e
I20260807 15:12:21.902704 13114 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d38713e0a7834cb39624cfd1d23d5ecb: Bootstrap starting.
I20260807 15:12:21.903484 13114 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d38713e0a7834cb39624cfd1d23d5ecb: Neither blocks nor log segments found. Creating new log.
I20260807 15:12:21.903793 13114 log.cc:824] T 00000000000000000000000000000000 P d38713e0a7834cb39624cfd1d23d5ecb: Log is configured to *not* fsync() on all Append() calls
I20260807 15:12:21.904518 13114 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d38713e0a7834cb39624cfd1d23d5ecb: No bootstrap required, opened a new log
I20260807 15:12:21.906559 13114 raft_consensus.cc:348] T 00000000000000000000000000000000 P d38713e0a7834cb39624cfd1d23d5ecb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d38713e0a7834cb39624cfd1d23d5ecb" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 41747 } }
I20260807 15:12:21.906756 13114 raft_consensus.cc:374] T 00000000000000000000000000000000 P d38713e0a7834cb39624cfd1d23d5ecb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
Aug 07 15:12:21 dist-test-slave-42z9 krb5kdc[12990](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115541, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.194.62@KRBTEST.COM
I20260807 15:12:21.906792 13114 raft_consensus.cc:729] T 00000000000000000000000000000000 P d38713e0a7834cb39624cfd1d23d5ecb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d38713e0a7834cb39624cfd1d23d5ecb, State: Initialized, Role: FOLLOWER
I20260807 15:12:21.906925 13114 consensus_queue.cc:260] T 00000000000000000000000000000000 P d38713e0a7834cb39624cfd1d23d5ecb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d38713e0a7834cb39624cfd1d23d5ecb" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 41747 } }
I20260807 15:12:21.907024 13114 raft_consensus.cc:388] T 00000000000000000000000000000000 P d38713e0a7834cb39624cfd1d23d5ecb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:12:21.907058 13114 raft_consensus.cc:482] T 00000000000000000000000000000000 P d38713e0a7834cb39624cfd1d23d5ecb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:12:21.907109 13114 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d38713e0a7834cb39624cfd1d23d5ecb [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:12:21.907711 13114 raft_consensus.cc:504] T 00000000000000000000000000000000 P d38713e0a7834cb39624cfd1d23d5ecb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d38713e0a7834cb39624cfd1d23d5ecb" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 41747 } }
I20260807 15:12:21.907809 13114 leader_election.cc:302] T 00000000000000000000000000000000 P d38713e0a7834cb39624cfd1d23d5ecb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d38713e0a7834cb39624cfd1d23d5ecb; no voters:
I20260807 15:12:21.907985 13114 leader_election.cc:288] T 00000000000000000000000000000000 P d38713e0a7834cb39624cfd1d23d5ecb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 15:12:21.908033 13122 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d38713e0a7834cb39624cfd1d23d5ecb [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:12:21.908172 13122 raft_consensus.cc:686] T 00000000000000000000000000000000 P d38713e0a7834cb39624cfd1d23d5ecb [term 1 LEADER]: Becoming Leader. State: Replica: d38713e0a7834cb39624cfd1d23d5ecb, State: Running, Role: LEADER
I20260807 15:12:21.908259 13122 consensus_queue.cc:237] T 00000000000000000000000000000000 P d38713e0a7834cb39624cfd1d23d5ecb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d38713e0a7834cb39624cfd1d23d5ecb" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 41747 } }
I20260807 15:12:21.908257 13114 sys_catalog.cc:564] T 00000000000000000000000000000000 P d38713e0a7834cb39624cfd1d23d5ecb [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:12:21.908593 13123 sys_catalog.cc:455] T 00000000000000000000000000000000 P d38713e0a7834cb39624cfd1d23d5ecb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d38713e0a7834cb39624cfd1d23d5ecb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d38713e0a7834cb39624cfd1d23d5ecb" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 41747 } } }
I20260807 15:12:21.908598 13124 sys_catalog.cc:455] T 00000000000000000000000000000000 P d38713e0a7834cb39624cfd1d23d5ecb [sys.catalog]: SysCatalogTable state changed. Reason: New leader d38713e0a7834cb39624cfd1d23d5ecb. Latest consensus state: current_term: 1 leader_uuid: "d38713e0a7834cb39624cfd1d23d5ecb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d38713e0a7834cb39624cfd1d23d5ecb" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 41747 } } }
I20260807 15:12:21.908671 13123 sys_catalog.cc:458] T 00000000000000000000000000000000 P d38713e0a7834cb39624cfd1d23d5ecb [sys.catalog]: This master's current role is: LEADER
I20260807 15:12:21.908846 13124 sys_catalog.cc:458] T 00000000000000000000000000000000 P d38713e0a7834cb39624cfd1d23d5ecb [sys.catalog]: This master's current role is: LEADER
I20260807 15:12:21.909006 13128 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:12:21.909629 13128 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:12:21.911284 13128 catalog_manager.cc:1347] Generated new cluster ID: 8ee51eba9b094ed2aa663ab23428e4bf
I20260807 15:12:21.911338 13128 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:12:21.912075 13119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:21.900203 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47756 (local address 127.29.194.62:41747)
0807 15:12:21.900363 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0807 15:12:21.900371 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 15:12:21.900397 (+ 26us) server_negotiation.cc:407] Connection header received
0807 15:12:21.900892 (+ 495us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:12:21.900898 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:12:21.900950 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:12:21.901070 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:12:21.901928 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:21.902894 (+ 966us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:21.903983 (+ 1089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:21.904683 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:21.904794 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:12:21.907366 (+ 2572us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:12:21.907371 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:12:21.907371 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:12:21.907383 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:12:21.910409 (+ 3026us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:12:21.910837 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:12:21.910840 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:12:21.910841 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:12:21.910954 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:12:21.911180 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:12:21.911180 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:12:21.911181 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:12:21.911503 (+ 322us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:12:21.911632 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0807 15:12:21.911734 (+ 102us) server_negotiation.cc:299] Negotiation successful
0807 15:12:21.911877 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":114,"thread_start_us":64,"threads_started":1}
I20260807 15:12:21.927064 13128 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:12:21.927873 13128 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:12:21.931835 13128 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d38713e0a7834cb39624cfd1d23d5ecb: Generated new TSK 0
I20260807 15:12:21.932098 13128 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.29.194.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.194.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.194.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.1.keytab.
Entry for principal kudu/127.29.194.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.1.keytab.
I20260807 15:12:21.960963 30472 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7fZSIL/build/release/bin/kudu
/tmp/dist-test-task7fZSIL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115480851507-30472-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115480851507-30472-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115480851507-30472-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115480851507-30472-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.194.1:0
--local_ip_for_outbound_sockets=127.29.194.1
--webserver_interface=127.29.194.1
--webserver_port=0
--tserver_master_addrs=127.29.194.62:41747
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.1.keytab
--principal=kudu/127.29.194.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.194.20:39625
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115480851507-30472-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115480851507-30472-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115480851507-30472-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:12:22.055100 13148 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:12:22.055358 13148 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:12:22.055384 13148 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:12:22.056613 13148 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:12:22.056664 13148 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:12:22.056692 13148 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.194.1
I20260807 15:12:22.058327 13148 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.194.20:39625
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115480851507-30472-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115480851507-30472-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.1.keytab
--principal=kudu/127.29.194.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.194.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115480851507-30472-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.194.1
--webserver_port=0
--tserver_master_addrs=127.29.194.62:41747
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13148
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.194.1
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115480851507-30472-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:12:22.058568 13148 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:12:22.058785 13148 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:12:22.064541 13156 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:12:22.064561 13158 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:12:22.064540 13155 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:12:22.065073 13148 server_base.cc:1034] running on GCE node
I20260807 15:12:22.065290 13148 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:12:22.065570 13148 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:12:22.066744 13148 hybrid_clock.cc:648] HybridClock initialized: now 1786115542066733 us; error 29 us; skew 500 ppm
Aug 07 15:12:22 dist-test-slave-42z9 krb5kdc[12990](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115542, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.194.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:12:22.069765 13148 init.cc:377] Logged in from keytab as kudu/127.29.194.1@KRBTEST.COM (short username kudu)
I20260807 15:12:22.070286 13148 webserver.cc:460] Webserver started at http://127.29.194.1:44263/ using document root <none> and password file <none>
I20260807 15:12:22.070504 13148 fs_manager.cc:359] Metadata directory not provided
I20260807 15:12:22.070550 13148 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:12:22.070677 13148 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:12:22.071970 13148 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115480851507-30472-0/minicluster-data/ts-0/data/instance:
uuid: "598eac33d88640429403d6594245d162"
format_stamp: "Formatted at 2026-08-07 15:12:22 on dist-test-slave-42z9"
server_key: "7ac9bcd2b702c4f9e0f57a0fc7a3ea51"
server_key_iv: "9d4c49662dd6e44ddeda0c5464666e94"
server_key_version: "encryptionkey@0"
I20260807 15:12:22.072348 13148 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115480851507-30472-0/minicluster-data/ts-0/wal/instance:
uuid: "598eac33d88640429403d6594245d162"
format_stamp: "Formatted at 2026-08-07 15:12:22 on dist-test-slave-42z9"
server_key: "7ac9bcd2b702c4f9e0f57a0fc7a3ea51"
server_key_iv: "9d4c49662dd6e44ddeda0c5464666e94"
server_key_version: "encryptionkey@0"
I20260807 15:12:22.073854 13148 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 15:12:22.074674 13170 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:12:22.074852 13148 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:12:22.074945 13148 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115480851507-30472-0/minicluster-data/ts-0/data,/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115480851507-30472-0/minicluster-data/ts-0/wal
uuid: "598eac33d88640429403d6594245d162"
format_stamp: "Formatted at 2026-08-07 15:12:22 on dist-test-slave-42z9"
server_key: "7ac9bcd2b702c4f9e0f57a0fc7a3ea51"
server_key_iv: "9d4c49662dd6e44ddeda0c5464666e94"
server_key_version: "encryptionkey@0"
I20260807 15:12:22.075008 13148 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115480851507-30472-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115480851507-30472-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115480851507-30472-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:12:22.086974 13148 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:12:22.090195 13148 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:12:22.090349 13148 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:12:22.090577 13148 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:12:22.090915 13148 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:12:22.090987 13148 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:12:22.091027 13148 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:12:22.091050 13148 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:12:22.097353 13148 rpc_server.cc:307] RPC server started. Bound to: 127.29.194.1:46295
I20260807 15:12:22.097431 13342 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.194.1:46295 every 8 connection(s)
I20260807 15:12:22.097780 13148 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115480851507-30472-0/minicluster-data/ts-0/data/info.pb
I20260807 15:12:22.099229 30472 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7fZSIL/build/release/bin/kudu as pid 13148
I20260807 15:12:22.099375 30472 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115480851507-30472-0/minicluster-data/ts-0/wal/instance
I20260807 15:12:22.099584 30472 external_mini_cluster.cc:1428] Setting key 50e396f89d28eed3cadf5025ed89c07b
Aug 07 15:12:22 dist-test-slave-42z9 krb5kdc[12990](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115542, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.194.1@KRBTEST.COM for kudu/127.29.194.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.29.194.2@KRBTEST.COM; defaulting to no policy
I20260807 15:12:22.109953 13119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:22.098961 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.194.1:33959 (local address 127.29.194.62:41747)
0807 15:12:22.099043 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0807 15:12:22.099045 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:12:22.099145 (+ 100us) server_negotiation.cc:407] Connection header received
0807 15:12:22.099770 (+ 625us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:12:22.099818 (+ 48us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:12:22.099850 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:12:22.099960 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:12:22.101266 (+ 1306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:22.101849 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:22.103015 (+ 1166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:22.103313 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:22.103398 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:12:22.106178 (+ 2780us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:12:22.106180 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:12:22.106180 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:12:22.106188 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:12:22.107900 (+ 1712us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:12:22.108569 (+ 669us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:12:22.108571 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:12:22.108571 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:12:22.108680 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:12:22.109056 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:12:22.109058 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:12:22.109058 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:12:22.109219 (+ 161us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:12:22.109297 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0807 15:12:22.109734 (+ 437us) server_negotiation.cc:299] Negotiation successful
0807 15:12:22.109866 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260807 15:12:22.110069 13347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:22.098785 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:41747 (local address 127.29.194.1:33959)
0807 15:12:22.098995 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0807 15:12:22.099043 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0807 15:12:22.099595 (+ 552us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:12:22.100062 (+ 467us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:12:22.100067 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:12:22.100834 (+ 767us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:12:22.101124 (+ 290us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:12:22.101132 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:12:22.101946 (+ 814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:12:22.101948 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:12:22.102918 (+ 970us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:12:22.102921 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:12:22.103445 (+ 524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:12:22.103446 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:12:22.103558 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:12:22.104128 (+ 570us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:12:22.104137 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:12:22.106046 (+ 1909us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:12:22.108054 (+ 2008us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:12:22.108067 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:12:22.108070 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:12:22.108411 (+ 341us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:12:22.108795 (+ 384us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:12:22.108796 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:12:22.108797 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:12:22.108972 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:12:22.109304 (+ 332us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:12:22.109312 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:12:22.109532 (+ 220us) client_negotiation.cc:769] Sending connection context
0807 15:12:22.109661 (+ 129us) client_negotiation.cc:240] Negotiation successful
0807 15:12:22.109846 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":98,"threads_started":1}
I20260807 15:12:22.110603 13344 heartbeater.cc:347] Connected to a master server at 127.29.194.62:41747
I20260807 15:12:22.110704 13344 heartbeater.cc:464] Registering TS with master...
I20260807 15:12:22.110881 13344 heartbeater.cc:511] Master 127.29.194.62:41747 requested a full tablet report, sending...
I20260807 15:12:22.111420 13056 ts_manager.cc:194] Registered new tserver with Master: 598eac33d88640429403d6594245d162 (127.29.194.1:46295)
I20260807 15:12:22.112032 13056 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.194.1@KRBTEST.COM'} at 127.29.194.1:33959
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.194.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.194.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.2.keytab.
Entry for principal kudu/127.29.194.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.2.keytab.
I20260807 15:12:22.129029 30472 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7fZSIL/build/release/bin/kudu
/tmp/dist-test-task7fZSIL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115480851507-30472-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115480851507-30472-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115480851507-30472-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115480851507-30472-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.194.2:0
--local_ip_for_outbound_sockets=127.29.194.2
--webserver_interface=127.29.194.2
--webserver_port=0
--tserver_master_addrs=127.29.194.62:41747
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.2.keytab
--principal=kudu/127.29.194.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.194.20:39625
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115480851507-30472-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115480851507-30472-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115480851507-30472-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:12:22.215607 13350 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:12:22.215826 13350 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:12:22.215849 13350 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:12:22.216987 13350 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:12:22.217027 13350 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:12:22.217053 13350 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.194.2
I20260807 15:12:22.218551 13350 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.194.20:39625
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115480851507-30472-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115480851507-30472-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.2.keytab
--principal=kudu/127.29.194.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.194.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115480851507-30472-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.194.2
--webserver_port=0
--tserver_master_addrs=127.29.194.62:41747
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13350
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.194.2
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115480851507-30472-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:12:22.218772 13350 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:12:22.218983 13350 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:12:22.224360 13366 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:12:22.224414 13362 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:12:22.224341 13368 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:12:22.224529 13350 server_base.cc:1034] running on GCE node
I20260807 15:12:22.225026 13350 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:12:22.225286 13350 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:12:22.226516 13350 hybrid_clock.cc:648] HybridClock initialized: now 1786115542226502 us; error 47 us; skew 500 ppm
Aug 07 15:12:22 dist-test-slave-42z9 krb5kdc[12990](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115542, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.194.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:12:22.229517 13350 init.cc:377] Logged in from keytab as kudu/127.29.194.2@KRBTEST.COM (short username kudu)
I20260807 15:12:22.229993 13350 webserver.cc:460] Webserver started at http://127.29.194.2:34079/ using document root <none> and password file <none>
I20260807 15:12:22.230232 13350 fs_manager.cc:359] Metadata directory not provided
I20260807 15:12:22.230283 13350 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:12:22.230412 13350 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:12:22.231474 13350 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115480851507-30472-0/minicluster-data/ts-1/data/instance:
uuid: "6e540c72b27542c4ba25c958f462d00b"
format_stamp: "Formatted at 2026-08-07 15:12:22 on dist-test-slave-42z9"
server_key: "246ddd4b2d2bd6b226a07a9fe8065847"
server_key_iv: "a184b5e99048f4d0a44b6862e1a25935"
server_key_version: "encryptionkey@0"
I20260807 15:12:22.231814 13350 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115480851507-30472-0/minicluster-data/ts-1/wal/instance:
uuid: "6e540c72b27542c4ba25c958f462d00b"
format_stamp: "Formatted at 2026-08-07 15:12:22 on dist-test-slave-42z9"
server_key: "246ddd4b2d2bd6b226a07a9fe8065847"
server_key_iv: "a184b5e99048f4d0a44b6862e1a25935"
server_key_version: "encryptionkey@0"
I20260807 15:12:22.233168 13350 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 15:12:22.233990 13380 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:12:22.234174 13350 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260807 15:12:22.234262 13350 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115480851507-30472-0/minicluster-data/ts-1/data,/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115480851507-30472-0/minicluster-data/ts-1/wal
uuid: "6e540c72b27542c4ba25c958f462d00b"
format_stamp: "Formatted at 2026-08-07 15:12:22 on dist-test-slave-42z9"
server_key: "246ddd4b2d2bd6b226a07a9fe8065847"
server_key_iv: "a184b5e99048f4d0a44b6862e1a25935"
server_key_version: "encryptionkey@0"
I20260807 15:12:22.234333 13350 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115480851507-30472-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115480851507-30472-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115480851507-30472-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:12:22.248301 13350 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:12:22.250854 13350 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:12:22.251013 13350 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:12:22.251261 13350 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:12:22.251592 13350 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:12:22.251628 13350 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:12:22.251657 13350 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:12:22.251673 13350 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:12:22.257370 13350 rpc_server.cc:307] RPC server started. Bound to: 127.29.194.2:41531
I20260807 15:12:22.257606 13554 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.194.2:41531 every 8 connection(s)
I20260807 15:12:22.257800 13350 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115480851507-30472-0/minicluster-data/ts-1/data/info.pb
Aug 07 15:12:22 dist-test-slave-42z9 krb5kdc[12990](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115542, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.194.2@KRBTEST.COM for kudu/127.29.194.62@KRBTEST.COM
I20260807 15:12:22.265568 30472 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7fZSIL/build/release/bin/kudu as pid 13350
I20260807 15:12:22.265662 30472 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115480851507-30472-0/minicluster-data/ts-1/wal/instance
I20260807 15:12:22.265849 30472 external_mini_cluster.cc:1428] Setting key 0e47f7610701fc980c8a50b5c22c726d
I20260807 15:12:22.269570 13119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:22.258617 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.194.2:60241 (local address 127.29.194.62:41747)
0807 15:12:22.258700 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0807 15:12:22.258702 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:12:22.258928 (+ 226us) server_negotiation.cc:407] Connection header received
0807 15:12:22.259524 (+ 596us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:12:22.259526 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:12:22.259544 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:12:22.259655 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:12:22.261099 (+ 1444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:22.261707 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:22.262887 (+ 1180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:22.263109 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:22.263176 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:12:22.266173 (+ 2997us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:12:22.266175 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:12:22.266175 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:12:22.266184 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:12:22.267628 (+ 1444us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:12:22.268194 (+ 566us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:12:22.268195 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:12:22.268196 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:12:22.268280 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:12:22.268569 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:12:22.268570 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:12:22.268570 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:12:22.268728 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:12:22.268802 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0807 15:12:22.269330 (+ 528us) server_negotiation.cc:299] Negotiation successful
0807 15:12:22.269475 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260807 15:12:22.269889 13557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:22.258509 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:41747 (local address 127.29.194.2:60241)
0807 15:12:22.258808 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0807 15:12:22.258843 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0807 15:12:22.259354 (+ 511us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:12:22.259720 (+ 366us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:12:22.259724 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:12:22.260560 (+ 836us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:12:22.260886 (+ 326us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:12:22.260895 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:12:22.261853 (+ 958us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:12:22.261855 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:12:22.262771 (+ 916us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:12:22.262775 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:12:22.263244 (+ 469us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:12:22.263245 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:12:22.263431 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:12:22.264209 (+ 778us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:12:22.264220 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:12:22.266080 (+ 1860us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:12:22.267728 (+ 1648us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:12:22.267740 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:12:22.267743 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:12:22.268093 (+ 350us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:12:22.268363 (+ 270us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:12:22.268364 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:12:22.268364 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:12:22.268507 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:12:22.268890 (+ 383us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:12:22.268904 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:12:22.269139 (+ 235us) client_negotiation.cc:769] Sending connection context
0807 15:12:22.269288 (+ 149us) client_negotiation.cc:240] Negotiation successful
0807 15:12:22.269563 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":263,"thread_start_us":228,"threads_started":1}
I20260807 15:12:22.270648 13555 heartbeater.cc:347] Connected to a master server at 127.29.194.62:41747
I20260807 15:12:22.270807 13555 heartbeater.cc:464] Registering TS with master...
I20260807 15:12:22.271054 13555 heartbeater.cc:511] Master 127.29.194.62:41747 requested a full tablet report, sending...
I20260807 15:12:22.271693 13056 ts_manager.cc:194] Registered new tserver with Master: 6e540c72b27542c4ba25c958f462d00b (127.29.194.2:41531)
I20260807 15:12:22.272192 13056 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.194.2@KRBTEST.COM'} at 127.29.194.2:60241
WARNING: no policy specified for kudu/127.29.194.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.194.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.194.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.3.keytab.
Entry for principal kudu/127.29.194.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.3.keytab.
I20260807 15:12:22.296636 30472 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7fZSIL/build/release/bin/kudu
/tmp/dist-test-task7fZSIL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115480851507-30472-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115480851507-30472-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115480851507-30472-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115480851507-30472-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.194.3:0
--local_ip_for_outbound_sockets=127.29.194.3
--webserver_interface=127.29.194.3
--webserver_port=0
--tserver_master_addrs=127.29.194.62:41747
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.3.keytab
--principal=kudu/127.29.194.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.194.20:39625
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115480851507-30472-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115480851507-30472-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115480851507-30472-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:12:22.392004 13561 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:12:22.392263 13561 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:12:22.392292 13561 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:12:22.393652 13561 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:12:22.393707 13561 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:12:22.393738 13561 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.194.3
I20260807 15:12:22.395452 13561 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.194.20:39625
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115480851507-30472-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115480851507-30472-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.3.keytab
--principal=kudu/127.29.194.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.194.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115480851507-30472-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.194.3
--webserver_port=0
--tserver_master_addrs=127.29.194.62:41747
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13561
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.194.3
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115480851507-30472-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:12:22.395757 13561 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:12:22.395970 13561 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:12:22.402734 13568 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:12:22.402808 13572 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:12:22.402734 13569 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:12:22.403239 13561 server_base.cc:1034] running on GCE node
I20260807 15:12:22.403479 13561 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:12:22.403740 13561 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:12:22.404930 13561 hybrid_clock.cc:648] HybridClock initialized: now 1786115542404909 us; error 32 us; skew 500 ppm
Aug 07 15:12:22 dist-test-slave-42z9 krb5kdc[12990](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115542, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.194.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:12:22.407789 13561 init.cc:377] Logged in from keytab as kudu/127.29.194.3@KRBTEST.COM (short username kudu)
I20260807 15:12:22.408335 13561 webserver.cc:460] Webserver started at http://127.29.194.3:33371/ using document root <none> and password file <none>
I20260807 15:12:22.408571 13561 fs_manager.cc:359] Metadata directory not provided
I20260807 15:12:22.408631 13561 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:12:22.408784 13561 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:12:22.410580 13561 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115480851507-30472-0/minicluster-data/ts-2/data/instance:
uuid: "8c20e7f869be40d9a67ac8dcba8a9f6f"
format_stamp: "Formatted at 2026-08-07 15:12:22 on dist-test-slave-42z9"
server_key: "12d52c0c00a92727358625c4c8b8f654"
server_key_iv: "49a68bc22f5cf1dcb1626d39392df3f3"
server_key_version: "encryptionkey@0"
I20260807 15:12:22.410936 13561 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115480851507-30472-0/minicluster-data/ts-2/wal/instance:
uuid: "8c20e7f869be40d9a67ac8dcba8a9f6f"
format_stamp: "Formatted at 2026-08-07 15:12:22 on dist-test-slave-42z9"
server_key: "12d52c0c00a92727358625c4c8b8f654"
server_key_iv: "49a68bc22f5cf1dcb1626d39392df3f3"
server_key_version: "encryptionkey@0"
I20260807 15:12:22.412302 13561 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260807 15:12:22.413156 13584 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:12:22.413475 13561 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:12:22.413569 13561 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115480851507-30472-0/minicluster-data/ts-2/data,/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115480851507-30472-0/minicluster-data/ts-2/wal
uuid: "8c20e7f869be40d9a67ac8dcba8a9f6f"
format_stamp: "Formatted at 2026-08-07 15:12:22 on dist-test-slave-42z9"
server_key: "12d52c0c00a92727358625c4c8b8f654"
server_key_iv: "49a68bc22f5cf1dcb1626d39392df3f3"
server_key_version: "encryptionkey@0"
I20260807 15:12:22.413632 13561 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115480851507-30472-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115480851507-30472-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115480851507-30472-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:12:22.438570 13561 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:12:22.441215 13561 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:12:22.441531 13561 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:12:22.441805 13561 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:12:22.442138 13561 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:12:22.442176 13561 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:12:22.442200 13561 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:12:22.442211 13561 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:12:22.449323 13561 rpc_server.cc:307] RPC server started. Bound to: 127.29.194.3:41943
I20260807 15:12:22.449404 13769 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.194.3:41943 every 8 connection(s)
I20260807 15:12:22.449777 13561 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115480851507-30472-0/minicluster-data/ts-2/data/info.pb
I20260807 15:12:22.455730 30472 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7fZSIL/build/release/bin/kudu as pid 13561
I20260807 15:12:22.455889 30472 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115480851507-30472-0/minicluster-data/ts-2/wal/instance
I20260807 15:12:22.456122 30472 external_mini_cluster.cc:1428] Setting key 38ff06262a830d0d1fac0feee292dc7e
Aug 07 15:12:22 dist-test-slave-42z9 krb5kdc[12990](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115542, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.194.3@KRBTEST.COM for kudu/127.29.194.62@KRBTEST.COM
I20260807 15:12:22.461179 13119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:22.450621 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.194.3:33569 (local address 127.29.194.62:41747)
0807 15:12:22.450678 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0807 15:12:22.450680 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:12:22.450903 (+ 223us) server_negotiation.cc:407] Connection header received
0807 15:12:22.451439 (+ 536us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:12:22.451440 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:12:22.451452 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:12:22.451511 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:12:22.452513 (+ 1002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:22.453067 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:22.454306 (+ 1239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:22.454505 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:22.454573 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:12:22.458074 (+ 3501us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:12:22.458076 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:12:22.458076 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:12:22.458084 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:12:22.459437 (+ 1353us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:12:22.459873 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:12:22.459873 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:12:22.459874 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:12:22.459926 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:12:22.460266 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:12:22.460268 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:12:22.460268 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:12:22.460478 (+ 210us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:12:22.460567 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0807 15:12:22.460967 (+ 400us) server_negotiation.cc:299] Negotiation successful
0807 15:12:22.461088 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260807 15:12:22.461388 13774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:22.450617 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:41747 (local address 127.29.194.3:33569)
0807 15:12:22.450764 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0807 15:12:22.450801 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0807 15:12:22.451313 (+ 512us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:12:22.451582 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:12:22.451586 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:12:22.452196 (+ 610us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:12:22.452404 (+ 208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:12:22.452408 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:12:22.453172 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:12:22.453173 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:12:22.454176 (+ 1003us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:12:22.454181 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:12:22.454689 (+ 508us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:12:22.454692 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:12:22.454953 (+ 261us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:12:22.455657 (+ 704us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:12:22.455676 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:12:22.457892 (+ 2216us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:12:22.459532 (+ 1640us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:12:22.459540 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:12:22.459542 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:12:22.459808 (+ 266us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:12:22.460009 (+ 201us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:12:22.460010 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:12:22.460010 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:12:22.460138 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:12:22.460594 (+ 456us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:12:22.460600 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:12:22.460764 (+ 164us) client_negotiation.cc:769] Sending connection context
0807 15:12:22.460910 (+ 146us) client_negotiation.cc:240] Negotiation successful
0807 15:12:22.461148 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":72,"threads_started":1}
I20260807 15:12:22.461910 13770 heartbeater.cc:347] Connected to a master server at 127.29.194.62:41747
I20260807 15:12:22.462010 13770 heartbeater.cc:464] Registering TS with master...
I20260807 15:12:22.462173 13770 heartbeater.cc:511] Master 127.29.194.62:41747 requested a full tablet report, sending...
I20260807 15:12:22.462551 13056 ts_manager.cc:194] Registered new tserver with Master: 8c20e7f869be40d9a67ac8dcba8a9f6f (127.29.194.3:41943)
I20260807 15:12:22.462931 13056 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.194.3@KRBTEST.COM'} at 127.29.194.3:33569
I20260807 15:12:22.469208 30472 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 15:12:22.477362 13119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:22.470688 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47760 (local address 127.29.194.62:41747)
0807 15:12:22.470754 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0807 15:12:22.470756 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:12:22.470860 (+ 104us) server_negotiation.cc:407] Connection header received
0807 15:12:22.470884 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:12:22.470885 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:12:22.470894 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:12:22.470931 (+ 37us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:12:22.471562 (+ 631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:22.472072 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:22.472826 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:22.473020 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:22.473081 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:12:22.474464 (+ 1383us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:12:22.474465 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:12:22.474465 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:12:22.474472 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:12:22.475808 (+ 1336us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:12:22.476197 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:12:22.476197 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:12:22.476198 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:12:22.476245 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:12:22.476741 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:12:22.476741 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:12:22.476741 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:12:22.476900 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:12:22.476979 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0807 15:12:22.477196 (+ 217us) server_negotiation.cc:299] Negotiation successful
0807 15:12:22.477286 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260807 15:12:22.479238 13056 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47760:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 15:12:22.480894 13056 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 15:12:22.488797 13791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:22.484416 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39156 (local address 127.29.194.2:41531)
0807 15:12:22.484710 (+ 294us) server_negotiation.cc:206] Beginning negotiation
0807 15:12:22.484713 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:12:22.484722 (+ 9us) server_negotiation.cc:407] Connection header received
0807 15:12:22.484757 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:12:22.484759 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:12:22.484771 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:12:22.484845 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:12:22.485155 (+ 310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:22.486302 (+ 1147us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:22.487867 (+ 1565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:22.488364 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:22.488428 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:12:22.488491 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0807 15:12:22.488650 (+ 159us) server_negotiation.cc:299] Negotiation successful
0807 15:12:22.488700 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":255,"thread_start_us":84,"threads_started":1}
I20260807 15:12:22.488811 13792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:22.484533 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.3:41943 (local address 127.0.0.1:50224)
0807 15:12:22.484820 (+ 287us) negotiation.cc:107] Waiting for socket to connect
0807 15:12:22.484830 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 15:12:22.484876 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:12:22.485225 (+ 349us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:12:22.485226 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:12:22.485228 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:12:22.485287 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:12:22.485289 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:12:22.486398 (+ 1109us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:12:22.486400 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:12:22.487543 (+ 1143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:12:22.487546 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:12:22.488503 (+ 957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:12:22.488505 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:12:22.488572 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:12:22.488578 (+ 6us) client_negotiation.cc:769] Sending connection context
0807 15:12:22.488609 (+ 31us) client_negotiation.cc:240] Negotiation successful
0807 15:12:22.488699 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":267,"mutex_wait_us":63,"thread_start_us":78,"threads_started":1}
I20260807 15:12:22.488934 13788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:22.484259 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.1:46295 (local address 127.0.0.1:55424)
0807 15:12:22.484525 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0807 15:12:22.484568 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0807 15:12:22.484631 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:12:22.484918 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:12:22.484921 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:12:22.484925 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:12:22.485023 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:12:22.485026 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:12:22.486420 (+ 1394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:12:22.486421 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:12:22.487766 (+ 1345us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:12:22.487769 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:12:22.488727 (+ 958us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:12:22.488728 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:12:22.488787 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:12:22.488791 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 15:12:22.488837 (+ 46us) client_negotiation.cc:240] Negotiation successful
0807 15:12:22.488875 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":236,"mutex_wait_us":63,"thread_start_us":73,"threads_started":1}
I20260807 15:12:22.488806 13787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:22.484254 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.2:41531 (local address 127.0.0.1:39156)
0807 15:12:22.484495 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0807 15:12:22.484537 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0807 15:12:22.484613 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:12:22.484918 (+ 305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:12:22.484921 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:12:22.484923 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:12:22.485080 (+ 157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:12:22.485082 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:12:22.486420 (+ 1338us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:12:22.486421 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:12:22.487766 (+ 1345us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:12:22.487769 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:12:22.488450 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:12:22.488450 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:12:22.488548 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:12:22.488561 (+ 13us) client_negotiation.cc:769] Sending connection context
0807 15:12:22.488624 (+ 63us) client_negotiation.cc:240] Negotiation successful
0807 15:12:22.488710 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":179,"mutex_wait_us":277,"thread_start_us":93,"threads_started":1}
I20260807 15:12:22.489073 13793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:22.484811 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50224 (local address 127.29.194.3:41943)
0807 15:12:22.485068 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0807 15:12:22.485072 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:12:22.485080 (+ 8us) server_negotiation.cc:407] Connection header received
0807 15:12:22.485118 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:12:22.485120 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:12:22.485135 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:12:22.485358 (+ 223us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:12:22.485414 (+ 56us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:22.486294 (+ 880us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:22.487642 (+ 1348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:22.488394 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:22.488850 (+ 456us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:12:22.488914 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0807 15:12:22.488941 (+ 27us) server_negotiation.cc:299] Negotiation successful
0807 15:12:22.489000 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":223,"thread_start_us":93,"threads_started":1}
I20260807 15:12:22.489531 13790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:22.484330 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55424 (local address 127.29.194.1:46295)
0807 15:12:22.484612 (+ 282us) server_negotiation.cc:206] Beginning negotiation
0807 15:12:22.484617 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:12:22.484628 (+ 11us) server_negotiation.cc:407] Connection header received
0807 15:12:22.484735 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:12:22.484737 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:12:22.484753 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:12:22.484823 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:12:22.485105 (+ 282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:22.486302 (+ 1197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:22.487861 (+ 1559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:22.488579 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:22.488639 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:12:22.488694 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0807 15:12:22.489032 (+ 338us) server_negotiation.cc:299] Negotiation successful
0807 15:12:22.489082 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":245,"thread_start_us":93,"threads_started":1}
I20260807 15:12:22.490013 13662 tablet_service.cc:1467] Processing CreateTablet for tablet de8f8e228fa3475f80d2ea8758880a91 (DEFAULT_TABLE table=test-table [id=61c4791bb03149228fb286d4388cfe93]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:12:22.490099 13239 tablet_service.cc:1467] Processing CreateTablet for tablet de8f8e228fa3475f80d2ea8758880a91 (DEFAULT_TABLE table=test-table [id=61c4791bb03149228fb286d4388cfe93]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:12:22.490447 13662 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet de8f8e228fa3475f80d2ea8758880a91. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:12:22.490442 13239 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet de8f8e228fa3475f80d2ea8758880a91. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:12:22.490522 13452 tablet_service.cc:1467] Processing CreateTablet for tablet de8f8e228fa3475f80d2ea8758880a91 (DEFAULT_TABLE table=test-table [id=61c4791bb03149228fb286d4388cfe93]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:12:22.490862 13452 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet de8f8e228fa3475f80d2ea8758880a91. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:12:22.493614 13794 tablet_bootstrap.cc:492] T de8f8e228fa3475f80d2ea8758880a91 P 6e540c72b27542c4ba25c958f462d00b: Bootstrap starting.
I20260807 15:12:22.493840 13800 tablet_bootstrap.cc:492] T de8f8e228fa3475f80d2ea8758880a91 P 598eac33d88640429403d6594245d162: Bootstrap starting.
I20260807 15:12:22.494513 13794 tablet_bootstrap.cc:654] T de8f8e228fa3475f80d2ea8758880a91 P 6e540c72b27542c4ba25c958f462d00b: Neither blocks nor log segments found. Creating new log.
I20260807 15:12:22.494513 13800 tablet_bootstrap.cc:654] T de8f8e228fa3475f80d2ea8758880a91 P 598eac33d88640429403d6594245d162: Neither blocks nor log segments found. Creating new log.
I20260807 15:12:22.495090 13800 log.cc:824] T de8f8e228fa3475f80d2ea8758880a91 P 598eac33d88640429403d6594245d162: Log is configured to *not* fsync() on all Append() calls
I20260807 15:12:22.495090 13794 log.cc:824] T de8f8e228fa3475f80d2ea8758880a91 P 6e540c72b27542c4ba25c958f462d00b: Log is configured to *not* fsync() on all Append() calls
I20260807 15:12:22.495466 13797 tablet_bootstrap.cc:492] T de8f8e228fa3475f80d2ea8758880a91 P 8c20e7f869be40d9a67ac8dcba8a9f6f: Bootstrap starting.
I20260807 15:12:22.495959 13800 tablet_bootstrap.cc:492] T de8f8e228fa3475f80d2ea8758880a91 P 598eac33d88640429403d6594245d162: No bootstrap required, opened a new log
I20260807 15:12:22.496029 13800 ts_tablet_manager.cc:1397] T de8f8e228fa3475f80d2ea8758880a91 P 598eac33d88640429403d6594245d162: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260807 15:12:22.496052 13797 tablet_bootstrap.cc:654] T de8f8e228fa3475f80d2ea8758880a91 P 8c20e7f869be40d9a67ac8dcba8a9f6f: Neither blocks nor log segments found. Creating new log.
I20260807 15:12:22.496160 13794 tablet_bootstrap.cc:492] T de8f8e228fa3475f80d2ea8758880a91 P 6e540c72b27542c4ba25c958f462d00b: No bootstrap required, opened a new log
I20260807 15:12:22.496239 13794 ts_tablet_manager.cc:1397] T de8f8e228fa3475f80d2ea8758880a91 P 6e540c72b27542c4ba25c958f462d00b: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260807 15:12:22.496646 13797 log.cc:824] T de8f8e228fa3475f80d2ea8758880a91 P 8c20e7f869be40d9a67ac8dcba8a9f6f: Log is configured to *not* fsync() on all Append() calls
I20260807 15:12:22.497854 13800 raft_consensus.cc:348] T de8f8e228fa3475f80d2ea8758880a91 P 598eac33d88640429403d6594245d162 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "598eac33d88640429403d6594245d162" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 46295 } } peers { permanent_uuid: "6e540c72b27542c4ba25c958f462d00b" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 41531 } } peers { permanent_uuid: "8c20e7f869be40d9a67ac8dcba8a9f6f" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 41943 } }
I20260807 15:12:22.497895 13794 raft_consensus.cc:348] T de8f8e228fa3475f80d2ea8758880a91 P 6e540c72b27542c4ba25c958f462d00b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "598eac33d88640429403d6594245d162" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 46295 } } peers { permanent_uuid: "6e540c72b27542c4ba25c958f462d00b" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 41531 } } peers { permanent_uuid: "8c20e7f869be40d9a67ac8dcba8a9f6f" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 41943 } }
I20260807 15:12:22.498008 13800 raft_consensus.cc:374] T de8f8e228fa3475f80d2ea8758880a91 P 598eac33d88640429403d6594245d162 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:12:22.498024 13794 raft_consensus.cc:374] T de8f8e228fa3475f80d2ea8758880a91 P 6e540c72b27542c4ba25c958f462d00b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:12:22.498051 13800 raft_consensus.cc:729] T de8f8e228fa3475f80d2ea8758880a91 P 598eac33d88640429403d6594245d162 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 598eac33d88640429403d6594245d162, State: Initialized, Role: FOLLOWER
I20260807 15:12:22.498057 13794 raft_consensus.cc:729] T de8f8e228fa3475f80d2ea8758880a91 P 6e540c72b27542c4ba25c958f462d00b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e540c72b27542c4ba25c958f462d00b, State: Initialized, Role: FOLLOWER
I20260807 15:12:22.498178 13794 consensus_queue.cc:260] T de8f8e228fa3475f80d2ea8758880a91 P 6e540c72b27542c4ba25c958f462d00b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "598eac33d88640429403d6594245d162" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 46295 } } peers { permanent_uuid: "6e540c72b27542c4ba25c958f462d00b" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 41531 } } peers { permanent_uuid: "8c20e7f869be40d9a67ac8dcba8a9f6f" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 41943 } }
I20260807 15:12:22.498178 13800 consensus_queue.cc:260] T de8f8e228fa3475f80d2ea8758880a91 P 598eac33d88640429403d6594245d162 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "598eac33d88640429403d6594245d162" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 46295 } } peers { permanent_uuid: "6e540c72b27542c4ba25c958f462d00b" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 41531 } } peers { permanent_uuid: "8c20e7f869be40d9a67ac8dcba8a9f6f" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 41943 } }
I20260807 15:12:22.498494 13794 ts_tablet_manager.cc:1428] T de8f8e228fa3475f80d2ea8758880a91 P 6e540c72b27542c4ba25c958f462d00b: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260807 15:12:22.498523 13344 heartbeater.cc:503] Master 127.29.194.62:41747 was elected leader, sending a full tablet report...
I20260807 15:12:22.498500 13555 heartbeater.cc:503] Master 127.29.194.62:41747 was elected leader, sending a full tablet report...
I20260807 15:12:22.498556 13800 ts_tablet_manager.cc:1428] T de8f8e228fa3475f80d2ea8758880a91 P 598eac33d88640429403d6594245d162: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260807 15:12:22.499239 13797 tablet_bootstrap.cc:492] T de8f8e228fa3475f80d2ea8758880a91 P 8c20e7f869be40d9a67ac8dcba8a9f6f: No bootstrap required, opened a new log
I20260807 15:12:22.499317 13797 ts_tablet_manager.cc:1397] T de8f8e228fa3475f80d2ea8758880a91 P 8c20e7f869be40d9a67ac8dcba8a9f6f: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260807 15:12:22.501204 13797 raft_consensus.cc:348] T de8f8e228fa3475f80d2ea8758880a91 P 8c20e7f869be40d9a67ac8dcba8a9f6f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "598eac33d88640429403d6594245d162" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 46295 } } peers { permanent_uuid: "6e540c72b27542c4ba25c958f462d00b" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 41531 } } peers { permanent_uuid: "8c20e7f869be40d9a67ac8dcba8a9f6f" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 41943 } }
I20260807 15:12:22.501355 13797 raft_consensus.cc:374] T de8f8e228fa3475f80d2ea8758880a91 P 8c20e7f869be40d9a67ac8dcba8a9f6f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:12:22.501389 13797 raft_consensus.cc:729] T de8f8e228fa3475f80d2ea8758880a91 P 8c20e7f869be40d9a67ac8dcba8a9f6f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c20e7f869be40d9a67ac8dcba8a9f6f, State: Initialized, Role: FOLLOWER
I20260807 15:12:22.501547 13797 consensus_queue.cc:260] T de8f8e228fa3475f80d2ea8758880a91 P 8c20e7f869be40d9a67ac8dcba8a9f6f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "598eac33d88640429403d6594245d162" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 46295 } } peers { permanent_uuid: "6e540c72b27542c4ba25c958f462d00b" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 41531 } } peers { permanent_uuid: "8c20e7f869be40d9a67ac8dcba8a9f6f" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 41943 } }
I20260807 15:12:22.501822 13797 ts_tablet_manager.cc:1428] T de8f8e228fa3475f80d2ea8758880a91 P 8c20e7f869be40d9a67ac8dcba8a9f6f: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260807 15:12:22.501847 13770 heartbeater.cc:503] Master 127.29.194.62:41747 was elected leader, sending a full tablet report...
W20260807 15:12:22.508898 13556 tablet.cc:2369] T de8f8e228fa3475f80d2ea8758880a91 P 6e540c72b27542c4ba25c958f462d00b: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 15:12:22.598763 13346 tablet.cc:2369] T de8f8e228fa3475f80d2ea8758880a91 P 598eac33d88640429403d6594245d162: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 15:12:22.627411 13807 raft_consensus.cc:482] T de8f8e228fa3475f80d2ea8758880a91 P 8c20e7f869be40d9a67ac8dcba8a9f6f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 15:12:22.627556 13807 raft_consensus.cc:504] T de8f8e228fa3475f80d2ea8758880a91 P 8c20e7f869be40d9a67ac8dcba8a9f6f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "598eac33d88640429403d6594245d162" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 46295 } } peers { permanent_uuid: "6e540c72b27542c4ba25c958f462d00b" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 41531 } } peers { permanent_uuid: "8c20e7f869be40d9a67ac8dcba8a9f6f" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 41943 } }
I20260807 15:12:22.627920 13807 leader_election.cc:288] T de8f8e228fa3475f80d2ea8758880a91 P 8c20e7f869be40d9a67ac8dcba8a9f6f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 598eac33d88640429403d6594245d162 (127.29.194.1:46295), 6e540c72b27542c4ba25c958f462d00b (127.29.194.2:41531)
I20260807 15:12:22.631027 13774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:22.628197 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.2:41531 (local address 127.29.194.3:44595)
0807 15:12:22.628258 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0807 15:12:22.628270 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 15:12:22.628381 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:12:22.628528 (+ 147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:12:22.628529 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:12:22.628530 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:12:22.628601 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:12:22.628603 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:12:22.629359 (+ 756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:12:22.629361 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:12:22.630207 (+ 846us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:12:22.630210 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:12:22.630765 (+ 555us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:12:22.630766 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:12:22.630847 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:12:22.630850 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 15:12:22.630898 (+ 48us) client_negotiation.cc:240] Negotiation successful
0807 15:12:22.630945 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37}
I20260807 15:12:22.631027 13791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:22.628311 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.194.3:44595 (local address 127.29.194.2:41531)
0807 15:12:22.628357 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0807 15:12:22.628359 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:12:22.628370 (+ 11us) server_negotiation.cc:407] Connection header received
0807 15:12:22.628459 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:12:22.628459 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:12:22.628469 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:12:22.628520 (+ 51us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:12:22.628656 (+ 136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:22.629268 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:22.630304 (+ 1036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:22.630692 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:22.630760 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:12:22.630769 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0807 15:12:22.630915 (+ 146us) server_negotiation.cc:299] Negotiation successful
0807 15:12:22.630960 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260807 15:12:22.631273 13790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:22.628336 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.194.3:36583 (local address 127.29.194.1:46295)
0807 15:12:22.628387 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0807 15:12:22.628389 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:12:22.628596 (+ 207us) server_negotiation.cc:407] Connection header received
0807 15:12:22.628652 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:12:22.628653 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:12:22.628661 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:12:22.628707 (+ 46us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:12:22.628851 (+ 144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:22.629599 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:22.630551 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:22.630914 (+ 363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:22.630978 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:12:22.630987 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0807 15:12:22.631150 (+ 163us) server_negotiation.cc:299] Negotiation successful
0807 15:12:22.631198 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260807 15:12:22.631449 13487 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "de8f8e228fa3475f80d2ea8758880a91" candidate_uuid: "8c20e7f869be40d9a67ac8dcba8a9f6f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e540c72b27542c4ba25c958f462d00b" is_pre_election: true
I20260807 15:12:22.631511 13810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:22.628265 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.1:46295 (local address 127.29.194.3:36583)
0807 15:12:22.628501 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0807 15:12:22.628510 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0807 15:12:22.628588 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:12:22.628716 (+ 128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:12:22.628717 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:12:22.628719 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:12:22.628788 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:12:22.628790 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:12:22.629681 (+ 891us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:12:22.629682 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:12:22.630469 (+ 787us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:12:22.630472 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:12:22.630985 (+ 513us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:12:22.630986 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:12:22.631068 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:12:22.631071 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 15:12:22.631396 (+ 325us) client_negotiation.cc:240] Negotiation successful
0807 15:12:22.631440 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":219,"thread_start_us":117,"threads_started":1}
I20260807 15:12:22.631628 13487 raft_consensus.cc:2454] T de8f8e228fa3475f80d2ea8758880a91 P 6e540c72b27542c4ba25c958f462d00b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c20e7f869be40d9a67ac8dcba8a9f6f in term 0.
I20260807 15:12:22.631816 13271 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "de8f8e228fa3475f80d2ea8758880a91" candidate_uuid: "8c20e7f869be40d9a67ac8dcba8a9f6f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "598eac33d88640429403d6594245d162" is_pre_election: true
I20260807 15:12:22.631873 13588 leader_election.cc:302] T de8f8e228fa3475f80d2ea8758880a91 P 8c20e7f869be40d9a67ac8dcba8a9f6f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e540c72b27542c4ba25c958f462d00b, 8c20e7f869be40d9a67ac8dcba8a9f6f; no voters:
I20260807 15:12:22.631953 13271 raft_consensus.cc:2454] T de8f8e228fa3475f80d2ea8758880a91 P 598eac33d88640429403d6594245d162 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c20e7f869be40d9a67ac8dcba8a9f6f in term 0.
I20260807 15:12:22.631994 13807 raft_consensus.cc:2781] T de8f8e228fa3475f80d2ea8758880a91 P 8c20e7f869be40d9a67ac8dcba8a9f6f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 15:12:22.632052 13807 raft_consensus.cc:482] T de8f8e228fa3475f80d2ea8758880a91 P 8c20e7f869be40d9a67ac8dcba8a9f6f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 15:12:22.632076 13807 raft_consensus.cc:3037] T de8f8e228fa3475f80d2ea8758880a91 P 8c20e7f869be40d9a67ac8dcba8a9f6f [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:12:22.632787 13807 raft_consensus.cc:504] T de8f8e228fa3475f80d2ea8758880a91 P 8c20e7f869be40d9a67ac8dcba8a9f6f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "598eac33d88640429403d6594245d162" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 46295 } } peers { permanent_uuid: "6e540c72b27542c4ba25c958f462d00b" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 41531 } } peers { permanent_uuid: "8c20e7f869be40d9a67ac8dcba8a9f6f" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 41943 } }
I20260807 15:12:22.632946 13807 leader_election.cc:288] T de8f8e228fa3475f80d2ea8758880a91 P 8c20e7f869be40d9a67ac8dcba8a9f6f [CANDIDATE]: Term 1 election: Requested vote from peers 598eac33d88640429403d6594245d162 (127.29.194.1:46295), 6e540c72b27542c4ba25c958f462d00b (127.29.194.2:41531)
I20260807 15:12:22.633106 13487 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "de8f8e228fa3475f80d2ea8758880a91" candidate_uuid: "8c20e7f869be40d9a67ac8dcba8a9f6f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e540c72b27542c4ba25c958f462d00b"
I20260807 15:12:22.633170 13271 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "de8f8e228fa3475f80d2ea8758880a91" candidate_uuid: "8c20e7f869be40d9a67ac8dcba8a9f6f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "598eac33d88640429403d6594245d162"
I20260807 15:12:22.633189 13487 raft_consensus.cc:3037] T de8f8e228fa3475f80d2ea8758880a91 P 6e540c72b27542c4ba25c958f462d00b [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:12:22.633263 13271 raft_consensus.cc:3037] T de8f8e228fa3475f80d2ea8758880a91 P 598eac33d88640429403d6594245d162 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:12:22.633984 13487 raft_consensus.cc:2454] T de8f8e228fa3475f80d2ea8758880a91 P 6e540c72b27542c4ba25c958f462d00b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c20e7f869be40d9a67ac8dcba8a9f6f in term 1.
I20260807 15:12:22.634032 13271 raft_consensus.cc:2454] T de8f8e228fa3475f80d2ea8758880a91 P 598eac33d88640429403d6594245d162 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c20e7f869be40d9a67ac8dcba8a9f6f in term 1.
I20260807 15:12:22.634171 13588 leader_election.cc:302] T de8f8e228fa3475f80d2ea8758880a91 P 8c20e7f869be40d9a67ac8dcba8a9f6f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e540c72b27542c4ba25c958f462d00b, 8c20e7f869be40d9a67ac8dcba8a9f6f; no voters:
I20260807 15:12:22.634280 13807 raft_consensus.cc:2781] T de8f8e228fa3475f80d2ea8758880a91 P 8c20e7f869be40d9a67ac8dcba8a9f6f [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:12:22.634439 13807 raft_consensus.cc:686] T de8f8e228fa3475f80d2ea8758880a91 P 8c20e7f869be40d9a67ac8dcba8a9f6f [term 1 LEADER]: Becoming Leader. State: Replica: 8c20e7f869be40d9a67ac8dcba8a9f6f, State: Running, Role: LEADER
I20260807 15:12:22.634564 13807 consensus_queue.cc:237] T de8f8e228fa3475f80d2ea8758880a91 P 8c20e7f869be40d9a67ac8dcba8a9f6f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "598eac33d88640429403d6594245d162" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 46295 } } peers { permanent_uuid: "6e540c72b27542c4ba25c958f462d00b" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 41531 } } peers { permanent_uuid: "8c20e7f869be40d9a67ac8dcba8a9f6f" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 41943 } }
I20260807 15:12:22.635488 13055 catalog_manager.cc:5495] T de8f8e228fa3475f80d2ea8758880a91 P 8c20e7f869be40d9a67ac8dcba8a9f6f reported cstate change: term changed from 0 to 1, leader changed from <none> to 8c20e7f869be40d9a67ac8dcba8a9f6f (127.29.194.3). New cstate: current_term: 1 leader_uuid: "8c20e7f869be40d9a67ac8dcba8a9f6f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "598eac33d88640429403d6594245d162" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 46295 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6e540c72b27542c4ba25c958f462d00b" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 41531 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8c20e7f869be40d9a67ac8dcba8a9f6f" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 41943 } health_report { overall_health: HEALTHY } } }
I20260807 15:12:22.642833 13793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:22.639853 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50234 (local address 127.29.194.3:41943)
0807 15:12:22.639906 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0807 15:12:22.639908 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:12:22.639916 (+ 8us) server_negotiation.cc:407] Connection header received
0807 15:12:22.639967 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:12:22.639968 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:12:22.639986 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:12:22.640063 (+ 77us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 15:12:22.640400 (+ 337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:22.640892 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:22.641825 (+ 933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:22.642053 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:22.642131 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:12:22.642377 (+ 246us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 15:12:22.642537 (+ 160us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 15:12:22.642641 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0807 15:12:22.642657 (+ 16us) server_negotiation.cc:299] Negotiation successful
0807 15:12:22.642692 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260807 15:12:22.645605 13271 raft_consensus.cc:1261] T de8f8e228fa3475f80d2ea8758880a91 P 598eac33d88640429403d6594245d162 [term 1 FOLLOWER]: Refusing update from remote peer 8c20e7f869be40d9a67ac8dcba8a9f6f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:12:22.645605 13487 raft_consensus.cc:1261] T de8f8e228fa3475f80d2ea8758880a91 P 6e540c72b27542c4ba25c958f462d00b [term 1 FOLLOWER]: Refusing update from remote peer 8c20e7f869be40d9a67ac8dcba8a9f6f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:12:22.646067 13811 consensus_queue.cc:1035] T de8f8e228fa3475f80d2ea8758880a91 P 8c20e7f869be40d9a67ac8dcba8a9f6f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e540c72b27542c4ba25c958f462d00b" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 41531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:12:22.646229 13807 consensus_queue.cc:1035] T de8f8e228fa3475f80d2ea8758880a91 P 8c20e7f869be40d9a67ac8dcba8a9f6f [LEADER]: Connected to new peer: Peer: permanent_uuid: "598eac33d88640429403d6594245d162" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 46295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:12:22.649199 13818 mvcc.cc:205] Tried to move back new op lower bound from 7315929262672228352 to 7315929262631800832. Current Snapshot: MvccSnapshot[applied={T|T < 7315929262672228352}]
I20260807 15:12:22.649991 13817 mvcc.cc:205] Tried to move back new op lower bound from 7315929262672228352 to 7315929262631800832. Current Snapshot: MvccSnapshot[applied={T|T < 7315929262672228352}]
I20260807 15:12:22.652356 13055 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47760:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 15:12:22.652477 13055 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47760:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 15:12:22.653378 13055 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d38713e0a7834cb39624cfd1d23d5ecb: Sending DeleteTablet for 3 replicas of tablet de8f8e228fa3475f80d2ea8758880a91
I20260807 15:12:22.653761 13662 tablet_service.cc:1514] Processing DeleteTablet for tablet de8f8e228fa3475f80d2ea8758880a91 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:12:22 UTC) from {username='kudu', principal='kudu/127.29.194.62@KRBTEST.COM'} at 127.0.0.1:50224
I20260807 15:12:22.653775 13239 tablet_service.cc:1514] Processing DeleteTablet for tablet de8f8e228fa3475f80d2ea8758880a91 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:12:22 UTC) from {username='kudu', principal='kudu/127.29.194.62@KRBTEST.COM'} at 127.0.0.1:55424
I20260807 15:12:22.654073 13829 tablet_replica.cc:330] T de8f8e228fa3475f80d2ea8758880a91 P 8c20e7f869be40d9a67ac8dcba8a9f6f: stopping tablet replica
I20260807 15:12:22.654184 13829 raft_consensus.cc:2229] T de8f8e228fa3475f80d2ea8758880a91 P 8c20e7f869be40d9a67ac8dcba8a9f6f [term 1 LEADER]: Raft consensus shutting down.
I20260807 15:12:22.654171 13452 tablet_service.cc:1514] Processing DeleteTablet for tablet de8f8e228fa3475f80d2ea8758880a91 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:12:22 UTC) from {username='kudu', principal='kudu/127.29.194.62@KRBTEST.COM'} at 127.0.0.1:39156
I20260807 15:12:22.654331 13829 raft_consensus.cc:2258] T de8f8e228fa3475f80d2ea8758880a91 P 8c20e7f869be40d9a67ac8dcba8a9f6f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:12:22.654408 30472 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7fZSIL/build/release/bin/kudu with pid 13148
I20260807 15:12:22.690227 13829 ts_tablet_manager.cc:1905] T de8f8e228fa3475f80d2ea8758880a91 P 8c20e7f869be40d9a67ac8dcba8a9f6f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:12:22.690500 13831 tablet_replica.cc:330] T de8f8e228fa3475f80d2ea8758880a91 P 6e540c72b27542c4ba25c958f462d00b: stopping tablet replica
I20260807 15:12:22.690728 13831 raft_consensus.cc:2229] T de8f8e228fa3475f80d2ea8758880a91 P 6e540c72b27542c4ba25c958f462d00b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 15:12:22.690912 13831 raft_consensus.cc:2258] T de8f8e228fa3475f80d2ea8758880a91 P 6e540c72b27542c4ba25c958f462d00b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:12:22.692706 13831 ts_tablet_manager.cc:1905] T de8f8e228fa3475f80d2ea8758880a91 P 6e540c72b27542c4ba25c958f462d00b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:12:22.693948 13831 ts_tablet_manager.cc:1918] T de8f8e228fa3475f80d2ea8758880a91 P 6e540c72b27542c4ba25c958f462d00b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 15:12:22.694034 13831 log.cc:1196] T de8f8e228fa3475f80d2ea8758880a91 P 6e540c72b27542c4ba25c958f462d00b: Deleting WAL directory at /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115480851507-30472-0/minicluster-data/ts-1/wal/wals/de8f8e228fa3475f80d2ea8758880a91
I20260807 15:12:22.694275 13831 ts_tablet_manager.cc:1939] T de8f8e228fa3475f80d2ea8758880a91 P 6e540c72b27542c4ba25c958f462d00b: Deleting consensus metadata
I20260807 15:12:22.694775 13829 ts_tablet_manager.cc:1918] T de8f8e228fa3475f80d2ea8758880a91 P 8c20e7f869be40d9a67ac8dcba8a9f6f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 15:12:22.694769 13036 catalog_manager.cc:4841] TS 6e540c72b27542c4ba25c958f462d00b (127.29.194.2:41531): tablet de8f8e228fa3475f80d2ea8758880a91 (table test-table [id=61c4791bb03149228fb286d4388cfe93]) successfully deleted
I20260807 15:12:22.694878 13829 log.cc:1196] T de8f8e228fa3475f80d2ea8758880a91 P 8c20e7f869be40d9a67ac8dcba8a9f6f: Deleting WAL directory at /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115480851507-30472-0/minicluster-data/ts-2/wal/wals/de8f8e228fa3475f80d2ea8758880a91
W20260807 15:12:22.694931 13037 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.29.194.1:46295 (error 108)
I20260807 15:12:22.695453 13829 ts_tablet_manager.cc:1939] T de8f8e228fa3475f80d2ea8758880a91 P 8c20e7f869be40d9a67ac8dcba8a9f6f: Deleting consensus metadata
I20260807 15:12:22.695509 13787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:22.695375 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.1:46295 (local address 127.0.0.1:55440)
0807 15:12:22.695428 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0807 15:12:22.695478 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.194.1:46295: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260807 15:12:22.695607 30472 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7fZSIL/build/release/bin/kudu with pid 13350
W20260807 15:12:22.695752 13037 catalog_manager.cc:4584] TS 598eac33d88640429403d6594245d162 (127.29.194.1:46295): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet de8f8e228fa3475f80d2ea8758880a91: Network error: Client connection negotiation failed: client connection to 127.29.194.1:46295: connect: Connection refused (error 111)
I20260807 15:12:22.728932 13037 catalog_manager.cc:4841] TS 8c20e7f869be40d9a67ac8dcba8a9f6f (127.29.194.3:41943): tablet de8f8e228fa3475f80d2ea8758880a91 (table test-table [id=61c4791bb03149228fb286d4388cfe93]) successfully deleted
I20260807 15:12:22.735525 30472 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7fZSIL/build/release/bin/kudu with pid 13561
I20260807 15:12:22.748940 13787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:22.748683 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.1:46295 (local address 127.0.0.1:55450)
0807 15:12:22.748808 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0807 15:12:22.748886 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.194.1:46295: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
I20260807 15:12:22.749531 13787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:22.749369 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.1:46295 (local address 127.0.0.1:55462)
0807 15:12:22.749422 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0807 15:12:22.749491 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.194.1:46295: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260807 15:12:22.749656 30472 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7fZSIL/build/release/bin/kudu with pid 13008
2026-08-07T15:12:22Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (2277 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115480851507-30472-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 15:12:22 dist-test-slave-42z9 krb5kdc[13835](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 15:12:22 dist-test-slave-42z9 krb5kdc[13835](info): set up 2 sockets
Aug 07 15:12:22 dist-test-slave-42z9 krb5kdc[13835](info): commencing operation
krb5kdc: starting...
W20260807 15:12:23.910427 30472 mini_kdc.cc:121] Time spent starting KDC: real 1.128s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115480851507-30472-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115480851507-30472-0/krb5kdc/test-user.keytab.
Aug 07 15:12:23 dist-test-slave-42z9 krb5kdc[13835](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115543, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T15:12:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T15:12:24Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.29.194.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.29.194.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.29.194.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115480851507-30472-0/krb5kdc/rangeradmin_127.29.194.20@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.29.194.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115480851507-30472-0/krb5kdc/rangeradmin_127.29.194.20@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.29.194.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.29.194.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.29.194.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115480851507-30472-0/krb5kdc/rangerlookup_127.29.194.20@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.29.194.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115480851507-30472-0/krb5kdc/rangerlookup_127.29.194.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.29.194.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.29.194.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.29.194.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115480851507-30472-0/krb5kdc/HTTP_127.29.194.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.29.194.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115480851507-30472-0/krb5kdc/HTTP_127.29.194.20@KRBTEST.COM.keytab.
I20260807 15:12:24.114326 30472 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115480851507-30472-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-task7fZSIL/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115480851507-30472-0/minicluster-data/postgres -l logfile start
2026-08-07 15:12:27.056 UTC [13887] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-07 15:12:27.056 UTC [13887] LOG: listening on IPv4 address "127.29.194.20", port 34997
2026-08-07 15:12:27.061 UTC [13887] LOG: listening on Unix socket "/tmp/.s.PGSQL.34997"
2026-08-07 15:12:27.084 UTC [13890] LOG: database system was shut down at 2026-08-07 15:12:25 UTC
2026-08-07 15:12:27.089 UTC [13887] LOG: database system is ready to accept connections
I20260807 15:12:28.175695 30472 mini_postgres.cc:96] Postgres bound to 34997
2026-08-07 15:12:28.184 UTC [13904] FATAL: database "slave" does not exist
127.29.194.20:34997 - accepting connections
I20260807 15:12:28.185767 30472 mini_ranger.cc:161] Starting Ranger...
I20260807 15:12:28.204249 30472 mini_ranger.cc:84] Created miniranger Postgres user
I20260807 15:12:28.793226 30472 mini_ranger.cc:87] Created ranger Postgres database
I20260807 15:12:28.793318 30472 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115480851507-30472-0/minicluster-data/ranger-admin
2026-08-07 15:12:29,109 [I] DB FLAVOR :POSTGRES
2026-08-07 15:12:29,110 [I] --------- Verifying Ranger DB connection ---------
2026-08-07 15:12:29,110 [I] Checking connection..
2026-08-07 15:12:29,110 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task7fZSIL/build/release/bin/postgresql.jar:/tmp/dist-test-task7fZSIL/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.194.20:34997/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-07 15:12:29,401 [I] Checking connection passed.
2026-08-07 15:12:29,401 [I] --------- Verifying version history table ---------
2026-08-07 15:12:29,402 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task7fZSIL/build/release/bin/postgresql.jar:/tmp/dist-test-task7fZSIL/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.194.20:34997/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-07 15:12:29,668 [I] Table x_db_version_h does not exist in database ranger
2026-08-07 15:12:29,669 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task7fZSIL/build/release/bin/postgresql.jar:/tmp/dist-test-task7fZSIL/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.194.20:34997/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-07 15:12:29,956 [I] Table x_db_version_h does not exist in database ranger
2026-08-07 15:12:29,957 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-07 15:12:29,957 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task7fZSIL/build/release/bin/postgresql.jar:/tmp/dist-test-task7fZSIL/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.194.20:34997/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-task7fZSIL/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-07 15:12:30.225 UTC [13997] WARNING: there is no transaction in progress
2026-08-07 15:12:30,234 [I] create_dbversion_catalog.sql file imported successfully
2026-08-07 15:12:30,234 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task7fZSIL/build/release/bin/postgresql.jar:/tmp/dist-test-task7fZSIL/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.194.20:34997/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-07 15:12:30,526 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-07 15:12:30,527 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-07 15:12:30,527 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task7fZSIL/build/release/bin/postgresql.jar:/tmp/dist-test-task7fZSIL/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.194.20:34997/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-07 15:12:30,804 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task7fZSIL/build/release/bin/postgresql.jar:/tmp/dist-test-task7fZSIL/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.194.20:34997/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-07 15:12:31,079 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task7fZSIL/build/release/bin/postgresql.jar:/tmp/dist-test-task7fZSIL/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.194.20:34997/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-42z9.c.gcp-upstream.internal','N') ;"
2026-08-07 15:12:31,342 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task7fZSIL/build/release/bin/postgresql.jar:/tmp/dist-test-task7fZSIL/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.194.20:34997/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-07 15:12:31,617 [I] Table x_portal_user does not exist in database ranger
2026-08-07 15:12:31,617 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task7fZSIL/build/release/bin/postgresql.jar:/tmp/dist-test-task7fZSIL/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.194.20:34997/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-07 15:12:31,900 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-07 15:12:31,900 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task7fZSIL/build/release/bin/postgresql.jar:/tmp/dist-test-task7fZSIL/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.194.20:34997/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-07 15:12:32,181 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task7fZSIL/build/release/bin/postgresql.jar:/tmp/dist-test-task7fZSIL/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.194.20:34997/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-07 15:12:32,457 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-07 15:12:32,457 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task7fZSIL/build/release/bin/postgresql.jar:/tmp/dist-test-task7fZSIL/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.194.20:34997/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-task7fZSIL/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-07 15:12:33.583 UTC [14170] WARNING: there is no transaction in progress
2026-08-07 15:12:33.602 UTC [14170] WARNING: there is no transaction in progress
2026-08-07 15:12:33.622 UTC [14170] WARNING: there is no transaction in progress
2026-08-07 15:12:33.640 UTC [14170] WARNING: there is no transaction in progress
2026-08-07 15:12:33.657 UTC [14170] WARNING: there is no transaction in progress
2026-08-07 15:12:33.676 UTC [14170] WARNING: there is no transaction in progress
2026-08-07 15:12:33.738 UTC [14170] WARNING: there is no transaction in progress
2026-08-07 15:12:33.746 UTC [14170] WARNING: there is no transaction in progress
2026-08-07 15:12:33.763 UTC [14170] WARNING: there is no transaction in progress
2026-08-07 15:12:33.776 UTC [14170] WARNING: there is no transaction in progress
2026-08-07 15:12:33.787 UTC [14170] WARNING: there is no transaction in progress
2026-08-07 15:12:33.802 UTC [14170] WARNING: there is no transaction in progress
2026-08-07 15:12:33.814 UTC [14170] WARNING: there is no transaction in progress
2026-08-07 15:12:33.821 UTC [14170] WARNING: there is no transaction in progress
2026-08-07 15:12:33.829 UTC [14170] WARNING: there is no transaction in progress
2026-08-07 15:12:33.836 UTC [14170] WARNING: there is no transaction in progress
2026-08-07 15:12:34,679 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-07 15:12:34,679 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task7fZSIL/build/release/bin/postgresql.jar:/tmp/dist-test-task7fZSIL/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.194.20:34997/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-07 15:12:34,951 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-07 15:12:34,951 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task7fZSIL/build/release/bin/postgresql.jar:/tmp/dist-test-task7fZSIL/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.194.20:34997/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-07 15:12:35,238 [I] Table x_portal_user already exists in database 'ranger'
2026-08-07 15:12:35,238 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task7fZSIL/build/release/bin/postgresql.jar:/tmp/dist-test-task7fZSIL/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.194.20:34997/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-07 15:12:35,520 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-07 15:12:35,520 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task7fZSIL/build/release/bin/postgresql.jar:/tmp/dist-test-task7fZSIL/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.194.20:34997/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-07 15:12:35,780 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task7fZSIL/build/release/bin/postgresql.jar:/tmp/dist-test-task7fZSIL/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.194.20:34997/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-07 15:12:36,038 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task7fZSIL/build/release/bin/postgresql.jar:/tmp/dist-test-task7fZSIL/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.194.20:34997/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-42z9.c.gcp-upstream.internal';"
2026-08-07 15:12:36,297 [I] CORE_DB_SCHEMA import status has been updated
2026-08-07 15:12:36,298 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task7fZSIL/build/release/bin/postgresql.jar:/tmp/dist-test-task7fZSIL/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.194.20:34997/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-07 15:12:36,586 [I] DB_PATCHES have already been applied
I20260807 15:12:36.591919 30472 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115480851507-30472-0/minicluster-data/ranger-admin:/tmp/dist-test-task7fZSIL/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-task7fZSIL/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-task7fZSIL/build/release/bin/postgresql.jar:/tmp/dist-test-task7fZSIL/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260807 15:12:36.592003 30472 mini_ranger.cc:193] Using host: 127.29.194.20
I20260807 15:12:36.596099 30472 mini_ranger.cc:227] Ranger admin URL: http://127.29.194.20:43117
Aug 07, 2026 3:12:36 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 07, 2026 3:12:36 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 07, 2026 3:12:37 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115480851507-30472-0/minicluster-data/ranger-admin/ews/webapp
Aug 07, 2026 3:12:37 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115480851507-30472-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 07, 2026 3:12:37 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115480851507-30472-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 07, 2026 3:12:37 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115480851507-30472-0/minicluster-data/ranger-admin/ews/webapp].
Aug 07, 2026 3:12:37 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260807 15:12:37.839962 30472 mini_ranger.cc:160] Time spent starting Ranger: real 9.654s user 0.006s sys 0.007s
Aug 07, 2026 3:13:00 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 07, 2026 3:13:00 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.MetricsREST
Aug 07, 2026 3:13:00 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 07, 2026 3:13:00 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 07, 2026 3:13:00 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 07, 2026 3:13:00 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 07, 2026 3:13:00 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 07, 2026 3:13:00 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 07, 2026 3:13:00 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 07, 2026 3:13:00 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 07, 2026 3:13:00 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 07, 2026 3:13:00 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 07, 2026 3:13:00 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 07, 2026 3:13:00 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 07, 2026 3:13:00 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 07, 2026 3:13:00 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 07, 2026 3:13:00 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 07, 2026 3:13:01 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260807 15:13:02.015825 30472 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.29.194.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.29.194.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.29.194.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115480851507-30472-0/krb5kdc/rangerkms_127.29.194.20@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.29.194.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115480851507-30472-0/krb5kdc/rangerkms_127.29.194.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.29.194.20@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.29.194.20@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.29.194.20@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115480851507-30472-0/krb5kdc/HTTP_127.29.194.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.29.194.20@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115480851507-30472-0/krb5kdc/HTTP_127.29.194.20@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 07 15:13:02 dist-test-slave-42z9 krb5kdc[13835](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115582, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260807 15:13:02.126184 30472 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260807 15:13:02.141053 30472 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260807 15:13:03.078385 30472 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260807 15:13:03.078477 30472 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115480851507-30472-0/minicluster-data/ranger-kms
I20260807 15:13:03.078490 30472 mini_ranger_kms.cc:227] Using postgres at 127.29.194.20:34997
2026-08-07 15:13:03,407 [I] DB FLAVOR :POSTGRES
2026-08-07 15:13:03,408 [I] --------- Verifying Ranger DB connection ---------
2026-08-07 15:13:03,408 [I] Checking connection
2026-08-07 15:13:03,408 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task7fZSIL/build/release/bin/postgresql.jar:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115480851507-30472-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.194.20:34997/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-07 15:13:03,683 [I] connection success
2026-08-07 15:13:03,684 [I] --------- Verifying Ranger DB tables ---------
2026-08-07 15:13:03,684 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-07 15:13:03,684 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task7fZSIL/build/release/bin/postgresql.jar:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115480851507-30472-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.194.20:34997/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-07 15:13:03,936 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-07 15:13:03,937 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-07 15:13:03,937 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-07 15:13:03,937 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task7fZSIL/build/release/bin/postgresql.jar:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115480851507-30472-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.194.20:34997/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115480851507-30472-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-07 15:13:04,219 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260807 15:13:04.722779 30472 mini_ranger_kms.cc:310] Created kms service
I20260807 15:13:04.919831 30472 mini_ranger_kms.cc:326] Created kudu user
I20260807 15:13:04.974716 30472 mini_ranger_kms.cc:343] Created rangerkms user
Aug 07 15:13:04 dist-test-slave-42z9 krb5kdc[13835](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115582, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.29.194.20@KRBTEST.COM
I20260807 15:13:05.397259 30472 mini_ranger_kms.cc:384] Added ranger policy
I20260807 15:13:05.397385 30472 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115480851507-30472-0/minicluster-data/ranger-kms:/tmp/dist-test-task7fZSIL/build/release/bin/postgresql.jar:/tmp/dist-test-task7fZSIL/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-task7fZSIL/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260807 15:13:05.397406 30472 mini_ranger_kms.cc:240] Using host: 127.29.194.20
I20260807 15:13:05.402246 30472 mini_ranger_kms.cc:276] Ranger KMS PID: 14443
I20260807 15:13:05.402386 30472 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.29.194.20:60841
Aug 07, 2026 3:13:06 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 07, 2026 3:13:06 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 07, 2026 3:13:06 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115480851507-30472-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 07, 2026 3:13:06 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115480851507-30472-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 07, 2026 3:13:07 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115480851507-30472-0/minicluster-data/ranger-kms/ews/webapp].
Aug 07, 2026 3:13:07 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 07 15:13:07 dist-test-slave-42z9 krb5kdc[13835](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786115587, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.29.194.20@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:13:07.886824 30472 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.761s user 0.016s sys 0.291s
I20260807 15:13:07.886957 30472 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260807 15:13:07.887043 30472 mini_ranger_kms.cc:401] 127.29.194.20:60841/kms/v1/keys
Aug 07 15:13:08 dist-test-slave-42z9 krb5kdc[13835](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115587, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.29.194.20@KRBTEST.COM for HTTP/127.29.194.20@KRBTEST.COM
Aug 07 15:13:08 dist-test-slave-42z9 krb5kdc[13835](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115587, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.29.194.20@KRBTEST.COM for HTTP/127.29.194.20@KRBTEST.COM
Aug 07 15:13:11 dist-test-slave-42z9 krb5kdc[13835](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115591, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.29.194.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.194.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.194.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.62.keytab.
Entry for principal kudu/127.29.194.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.62.keytab.
I20260807 15:13:11.149807 30472 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7fZSIL/build/release/bin/kudu
/tmp/dist-test-task7fZSIL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115480851507-30472-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115480851507-30472-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115480851507-30472-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115480851507-30472-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.194.62:37499
--webserver_interface=127.29.194.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.62.keytab
--principal=kudu/127.29.194.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.194.20:44037
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.194.62:37499
--ranger_config_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115480851507-30472-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.29.194.20:60841/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115480851507-30472-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115480851507-30472-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115480851507-30472-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:13:11.251940 14514 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:13:11.252161 14514 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:13:11.252192 14514 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:13:11.253369 14514 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:13:11.253412 14514 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:13:11.253463 14514 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:13:11.253509 14514 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:13:11.253537 14514 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:13:11.255283 14514 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.194.20:44037
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115480851507-30472-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115480851507-30472-0/minicluster-data/master-0/wal
--ranger_kms_url=127.29.194.20:60841/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.29.194.62:37499
--ranger_config_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115480851507-30472-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.62.keytab
--principal=kudu/127.29.194.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.194.62:37499
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115480851507-30472-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.194.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14514
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115480851507-30472-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:13:11.255604 14514 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:13:11.255877 14514 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:13:11.261777 14523 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:13:11.261869 14514 server_base.cc:1034] running on GCE node
W20260807 15:13:11.261782 14526 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:13:11.261817 14524 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:13:11.262437 14514 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:13:11.262828 14514 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:13:11.264015 14514 hybrid_clock.cc:648] HybridClock initialized: now 1786115591263991 us; error 41 us; skew 500 ppm
Aug 07 15:13:11 dist-test-slave-42z9 krb5kdc[13835](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115591, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.194.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:13:11.272485 14514 init.cc:377] Logged in from keytab as kudu/127.29.194.62@KRBTEST.COM (short username kudu)
I20260807 15:13:11.273166 14514 webserver.cc:460] Webserver started at http://127.29.194.62:43465/ using document root <none> and password file <none>
I20260807 15:13:11.273497 14514 fs_manager.cc:359] Metadata directory not provided
I20260807 15:13:11.273558 14514 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:13:11.273696 14514 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 15:13:11 dist-test-slave-42z9 krb5kdc[13835](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115591, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.194.62@KRBTEST.COM for HTTP/127.29.194.20@KRBTEST.COM
I20260807 15:13:11.321556 14514 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115480851507-30472-0/minicluster-data/master-0/data/instance:
uuid: "de41beabaa564ca294874d6874de509e"
format_stamp: "Formatted at 2026-08-07 15:13:11 on dist-test-slave-42z9"
server_key: "bc1887e969e968c034109f012fc53e71"
server_key_iv: "21c76fc93c3bb8bce337ca60f8f133d2"
server_key_version: "kuduclusterkey@0"
I20260807 15:13:11.322000 14514 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115480851507-30472-0/minicluster-data/master-0/wal/instance:
uuid: "de41beabaa564ca294874d6874de509e"
format_stamp: "Formatted at 2026-08-07 15:13:11 on dist-test-slave-42z9"
server_key: "bc1887e969e968c034109f012fc53e71"
server_key_iv: "21c76fc93c3bb8bce337ca60f8f133d2"
server_key_version: "kuduclusterkey@0"
I20260807 15:13:11.323756 14514 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260807 15:13:11.340747 14536 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:13:11.341080 14514 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:13:11.341195 14514 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115480851507-30472-0/minicluster-data/master-0/data,/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115480851507-30472-0/minicluster-data/master-0/wal
uuid: "de41beabaa564ca294874d6874de509e"
format_stamp: "Formatted at 2026-08-07 15:13:11 on dist-test-slave-42z9"
server_key: "bc1887e969e968c034109f012fc53e71"
server_key_iv: "21c76fc93c3bb8bce337ca60f8f133d2"
server_key_version: "kuduclusterkey@0"
I20260807 15:13:11.341300 14514 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115480851507-30472-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115480851507-30472-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115480851507-30472-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:13:11.350050 14514 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:13:11.351986 14514 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:13:11.352137 14514 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:13:11.358047 14514 rpc_server.cc:307] RPC server started. Bound to: 127.29.194.62:37499
I20260807 15:13:11.358119 14611 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.194.62:37499 every 8 connection(s)
I20260807 15:13:11.358551 14514 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115480851507-30472-0/minicluster-data/master-0/data/info.pb
I20260807 15:13:11.359453 14613 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:13:11.362198 30472 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7fZSIL/build/release/bin/kudu as pid 14514
I20260807 15:13:11.362262 14613 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P de41beabaa564ca294874d6874de509e: Bootstrap starting.
I20260807 15:13:11.362318 30472 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115480851507-30472-0/minicluster-data/master-0/wal/instance
I20260807 15:13:11.362526 30472 external_mini_cluster.cc:1428] Setting key 9632adc343c342ea1e3ab52b05ef145b
I20260807 15:13:11.363269 14613 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P de41beabaa564ca294874d6874de509e: Neither blocks nor log segments found. Creating new log.
I20260807 15:13:11.363788 14613 log.cc:824] T 00000000000000000000000000000000 P de41beabaa564ca294874d6874de509e: Log is configured to *not* fsync() on all Append() calls
I20260807 15:13:11.364846 14613 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P de41beabaa564ca294874d6874de509e: No bootstrap required, opened a new log
I20260807 15:13:11.367632 14613 raft_consensus.cc:348] T 00000000000000000000000000000000 P de41beabaa564ca294874d6874de509e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de41beabaa564ca294874d6874de509e" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 37499 } }
I20260807 15:13:11.367815 14613 raft_consensus.cc:374] T 00000000000000000000000000000000 P de41beabaa564ca294874d6874de509e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:13:11.367846 14613 raft_consensus.cc:729] T 00000000000000000000000000000000 P de41beabaa564ca294874d6874de509e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de41beabaa564ca294874d6874de509e, State: Initialized, Role: FOLLOWER
I20260807 15:13:11.368010 14613 consensus_queue.cc:260] T 00000000000000000000000000000000 P de41beabaa564ca294874d6874de509e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de41beabaa564ca294874d6874de509e" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 37499 } }
I20260807 15:13:11.368085 14613 raft_consensus.cc:388] T 00000000000000000000000000000000 P de41beabaa564ca294874d6874de509e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:13:11.368113 14613 raft_consensus.cc:482] T 00000000000000000000000000000000 P de41beabaa564ca294874d6874de509e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:13:11.368172 14613 raft_consensus.cc:3037] T 00000000000000000000000000000000 P de41beabaa564ca294874d6874de509e [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:13:11.368898 14613 raft_consensus.cc:504] T 00000000000000000000000000000000 P de41beabaa564ca294874d6874de509e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de41beabaa564ca294874d6874de509e" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 37499 } }
I20260807 15:13:11.369019 14613 leader_election.cc:302] T 00000000000000000000000000000000 P de41beabaa564ca294874d6874de509e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: de41beabaa564ca294874d6874de509e; no voters:
I20260807 15:13:11.369231 14613 leader_election.cc:288] T 00000000000000000000000000000000 P de41beabaa564ca294874d6874de509e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 15:13:11.369371 14620 raft_consensus.cc:2781] T 00000000000000000000000000000000 P de41beabaa564ca294874d6874de509e [term 1 FOLLOWER]: Leader election won for term 1
Aug 07 15:13:11 dist-test-slave-42z9 krb5kdc[13835](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115591, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.194.62@KRBTEST.COM
I20260807 15:13:11.369545 14613 sys_catalog.cc:564] T 00000000000000000000000000000000 P de41beabaa564ca294874d6874de509e [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:13:11.369597 14620 raft_consensus.cc:686] T 00000000000000000000000000000000 P de41beabaa564ca294874d6874de509e [term 1 LEADER]: Becoming Leader. State: Replica: de41beabaa564ca294874d6874de509e, State: Running, Role: LEADER
I20260807 15:13:11.369685 14620 consensus_queue.cc:237] T 00000000000000000000000000000000 P de41beabaa564ca294874d6874de509e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de41beabaa564ca294874d6874de509e" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 37499 } }
I20260807 15:13:11.370186 14621 sys_catalog.cc:455] T 00000000000000000000000000000000 P de41beabaa564ca294874d6874de509e [sys.catalog]: SysCatalogTable state changed. Reason: New leader de41beabaa564ca294874d6874de509e. Latest consensus state: current_term: 1 leader_uuid: "de41beabaa564ca294874d6874de509e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de41beabaa564ca294874d6874de509e" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 37499 } } }
I20260807 15:13:11.370289 14621 sys_catalog.cc:458] T 00000000000000000000000000000000 P de41beabaa564ca294874d6874de509e [sys.catalog]: This master's current role is: LEADER
I20260807 15:13:11.370039 14620 sys_catalog.cc:455] T 00000000000000000000000000000000 P de41beabaa564ca294874d6874de509e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "de41beabaa564ca294874d6874de509e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de41beabaa564ca294874d6874de509e" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 37499 } } }
I20260807 15:13:11.370554 14620 sys_catalog.cc:458] T 00000000000000000000000000000000 P de41beabaa564ca294874d6874de509e [sys.catalog]: This master's current role is: LEADER
I20260807 15:13:11.371985 14613 ranger_client.cc:318] Using new properties file: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115480851507-30472-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260807 15:13:11.372042 14641 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:13:11.372668 14641 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:13:11.374235 14618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:13:11.363120 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50750 (local address 127.29.194.62:37499)
0807 15:13:11.363323 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0807 15:13:11.363332 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 15:13:11.363365 (+ 33us) server_negotiation.cc:407] Connection header received
0807 15:13:11.364037 (+ 672us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:13:11.364048 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:13:11.364117 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:13:11.364276 (+ 159us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:13:11.364999 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:13:11.366140 (+ 1141us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:13:11.366858 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:13:11.367366 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:13:11.367478 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:13:11.370102 (+ 2624us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:13:11.370106 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:13:11.370107 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:13:11.370119 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:13:11.372242 (+ 2123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:13:11.372694 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:13:11.372697 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:13:11.372697 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:13:11.372800 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:13:11.373015 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:13:11.373016 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:13:11.373016 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:13:11.373300 (+ 284us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:13:11.373826 (+ 526us) server_negotiation.cc:1035] Waiting for connection context
0807 15:13:11.373904 (+ 78us) server_negotiation.cc:299] Negotiation successful
0807 15:13:11.374036 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":156,"thread_start_us":79,"threads_started":1}
I20260807 15:13:11.374608 14641 catalog_manager.cc:1347] Generated new cluster ID: 72b3c461d59b40c4b05fa7db4e2dda90
I20260807 15:13:11.374651 14641 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:13:11.389695 14641 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:13:11.390560 14641 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:13:11.402206 14641 catalog_manager.cc:5868] T 00000000000000000000000000000000 P de41beabaa564ca294874d6874de509e: Generated new TSK 0
I20260807 15:13:11.402534 14641 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.29.194.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.194.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.194.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.1.keytab.
Entry for principal kudu/127.29.194.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.1.keytab.
I20260807 15:13:12.267885 30472 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7fZSIL/build/release/bin/kudu
/tmp/dist-test-task7fZSIL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115480851507-30472-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115480851507-30472-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115480851507-30472-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115480851507-30472-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.29.194.20:60841/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.194.1:0
--local_ip_for_outbound_sockets=127.29.194.1
--webserver_interface=127.29.194.1
--webserver_port=0
--tserver_master_addrs=127.29.194.62:37499
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.1.keytab
--principal=kudu/127.29.194.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.194.20:44037
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115480851507-30472-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115480851507-30472-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115480851507-30472-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:13:12.372444 14685 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:13:12.372668 14685 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:13:12.372692 14685 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:13:12.373981 14685 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:13:12.374032 14685 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:13:12.374060 14685 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.194.1
I20260807 15:13:12.375665 14685 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.194.20:44037
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115480851507-30472-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115480851507-30472-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.29.194.20:60841/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.1.keytab
--principal=kudu/127.29.194.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.194.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115480851507-30472-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.194.1
--webserver_port=0
--tserver_master_addrs=127.29.194.62:37499
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14685
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.194.1
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115480851507-30472-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:13:12.375916 14685 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:13:12.376179 14685 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:13:12.381743 14694 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:13:12.381937 14691 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:13:12.382217 14692 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:13:12.382376 14685 server_base.cc:1034] running on GCE node
I20260807 15:13:12.382611 14685 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:13:12.382848 14685 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:13:12.386077 14685 hybrid_clock.cc:648] HybridClock initialized: now 1786115592386055 us; error 33 us; skew 500 ppm
Aug 07 15:13:12 dist-test-slave-42z9 krb5kdc[13835](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115592, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.194.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:13:12.389096 14685 init.cc:377] Logged in from keytab as kudu/127.29.194.1@KRBTEST.COM (short username kudu)
I20260807 15:13:12.389632 14685 webserver.cc:460] Webserver started at http://127.29.194.1:42387/ using document root <none> and password file <none>
I20260807 15:13:12.389883 14685 fs_manager.cc:359] Metadata directory not provided
I20260807 15:13:12.389933 14685 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:13:12.390121 14685 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 15:13:12 dist-test-slave-42z9 krb5kdc[13835](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115592, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.194.1@KRBTEST.COM for HTTP/127.29.194.20@KRBTEST.COM
I20260807 15:13:12.402058 14685 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115480851507-30472-0/minicluster-data/ts-0/data/instance:
uuid: "2b2738d42cfd4be5865540ff0ec11dc9"
format_stamp: "Formatted at 2026-08-07 15:13:12 on dist-test-slave-42z9"
server_key: "435d305652c1af838f75d9a5622d2e1b"
server_key_iv: "b60b9b88a9d477d4d606a889b102a153"
server_key_version: "kuduclusterkey@0"
I20260807 15:13:12.402431 14685 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115480851507-30472-0/minicluster-data/ts-0/wal/instance:
uuid: "2b2738d42cfd4be5865540ff0ec11dc9"
format_stamp: "Formatted at 2026-08-07 15:13:12 on dist-test-slave-42z9"
server_key: "435d305652c1af838f75d9a5622d2e1b"
server_key_iv: "b60b9b88a9d477d4d606a889b102a153"
server_key_version: "kuduclusterkey@0"
I20260807 15:13:12.405256 14685 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.001s
I20260807 15:13:12.420887 14704 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:13:12.421139 14685 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.002s sys 0.000s
I20260807 15:13:12.421247 14685 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115480851507-30472-0/minicluster-data/ts-0/data,/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115480851507-30472-0/minicluster-data/ts-0/wal
uuid: "2b2738d42cfd4be5865540ff0ec11dc9"
format_stamp: "Formatted at 2026-08-07 15:13:12 on dist-test-slave-42z9"
server_key: "435d305652c1af838f75d9a5622d2e1b"
server_key_iv: "b60b9b88a9d477d4d606a889b102a153"
server_key_version: "kuduclusterkey@0"
I20260807 15:13:12.421361 14685 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115480851507-30472-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115480851507-30472-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115480851507-30472-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:13:12.451939 14685 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:13:12.454285 14685 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:13:12.454452 14685 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:13:12.454757 14685 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:13:12.455188 14685 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:13:12.455281 14685 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:13:12.455389 14685 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:13:12.455467 14685 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:13:12.465914 14685 rpc_server.cc:307] RPC server started. Bound to: 127.29.194.1:44229
I20260807 15:13:12.466153 14881 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.194.1:44229 every 8 connection(s)
I20260807 15:13:12.466358 14685 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115480851507-30472-0/minicluster-data/ts-0/data/info.pb
Aug 07 15:13:12 dist-test-slave-42z9 krb5kdc[13835](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115592, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.194.1@KRBTEST.COM for kudu/127.29.194.62@KRBTEST.COM
I20260807 15:13:12.475545 30472 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7fZSIL/build/release/bin/kudu as pid 14685
I20260807 15:13:12.475648 30472 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115480851507-30472-0/minicluster-data/ts-0/wal/instance
I20260807 15:13:12.475839 30472 external_mini_cluster.cc:1428] Setting key 69771a7c78eb85a9a55ff38f48070431
I20260807 15:13:12.479703 14887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:13:12.467410 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.194.1:39929 (local address 127.29.194.62:37499)
0807 15:13:12.468731 (+ 1321us) server_negotiation.cc:206] Beginning negotiation
0807 15:13:12.468733 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:13:12.468744 (+ 11us) server_negotiation.cc:407] Connection header received
0807 15:13:12.468771 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:13:12.468772 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:13:12.468785 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:13:12.468859 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:13:12.469764 (+ 905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:13:12.470696 (+ 932us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:13:12.472264 (+ 1568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:13:12.472579 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:13:12.472661 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:13:12.475571 (+ 2910us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:13:12.475573 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:13:12.475573 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:13:12.475582 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:13:12.477879 (+ 2297us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:13:12.478483 (+ 604us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:13:12.478484 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:13:12.478484 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:13:12.478551 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:13:12.478872 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:13:12.478873 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:13:12.478874 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:13:12.479044 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:13:12.479121 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0807 15:13:12.479490 (+ 369us) server_negotiation.cc:299] Negotiation successful
0807 15:13:12.479593 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1293,"thread_start_us":84,"threads_started":1}
I20260807 15:13:12.479849 14884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:13:12.467327 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:37499 (local address 127.29.194.1:39929)
0807 15:13:12.467604 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0807 15:13:12.467656 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0807 15:13:12.468503 (+ 847us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:13:12.468994 (+ 491us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:13:12.468999 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:13:12.469287 (+ 288us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:13:12.469646 (+ 359us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:13:12.469653 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:13:12.470815 (+ 1162us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:13:12.470816 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:13:12.472162 (+ 1346us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:13:12.472165 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:13:12.472683 (+ 518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:13:12.472684 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:13:12.472839 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:13:12.473522 (+ 683us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:13:12.473532 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:13:12.475440 (+ 1908us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:13:12.478016 (+ 2576us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:13:12.478027 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:13:12.478031 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:13:12.478384 (+ 353us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:13:12.478615 (+ 231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:13:12.478616 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:13:12.478616 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:13:12.478780 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:13:12.479120 (+ 340us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:13:12.479126 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:13:12.479308 (+ 182us) client_negotiation.cc:769] Sending connection context
0807 15:13:12.479452 (+ 144us) client_negotiation.cc:240] Negotiation successful
0807 15:13:12.479634 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":106,"threads_started":1}
I20260807 15:13:12.480450 14882 heartbeater.cc:347] Connected to a master server at 127.29.194.62:37499
I20260807 15:13:12.480566 14882 heartbeater.cc:464] Registering TS with master...
I20260807 15:13:12.480763 14882 heartbeater.cc:511] Master 127.29.194.62:37499 requested a full tablet report, sending...
I20260807 15:13:12.481415 14565 ts_manager.cc:194] Registered new tserver with Master: 2b2738d42cfd4be5865540ff0ec11dc9 (127.29.194.1:44229)
I20260807 15:13:12.482357 14565 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.194.1@KRBTEST.COM'} at 127.29.194.1:39929
WARNING: no policy specified for kudu/127.29.194.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.194.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.194.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.2.keytab.
Entry for principal kudu/127.29.194.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.2.keytab.
I20260807 15:13:12.509248 30472 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7fZSIL/build/release/bin/kudu
/tmp/dist-test-task7fZSIL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115480851507-30472-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115480851507-30472-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115480851507-30472-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115480851507-30472-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.29.194.20:60841/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.194.2:0
--local_ip_for_outbound_sockets=127.29.194.2
--webserver_interface=127.29.194.2
--webserver_port=0
--tserver_master_addrs=127.29.194.62:37499
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.2.keytab
--principal=kudu/127.29.194.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.194.20:44037
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115480851507-30472-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115480851507-30472-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115480851507-30472-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:13:12.643111 14894 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:13:12.643496 14894 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:13:12.643589 14894 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:13:12.645627 14894 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:13:12.645699 14894 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:13:12.645742 14894 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.194.2
Aug 07 15:13:12 dist-test-slave-42z9 krb5kdc[13835](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786115592, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.194.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:13:12.648653 14894 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.194.20:44037
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115480851507-30472-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115480851507-30472-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.29.194.20:60841/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.2.keytab
--principal=kudu/127.29.194.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.194.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115480851507-30472-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.194.2
--webserver_port=0
--tserver_master_addrs=127.29.194.62:37499
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14894
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.194.2
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115480851507-30472-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:13:12.648988 14894 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:13:12.649283 14894 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:13:12.657224 14905 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:13:12.657579 14910 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:13:12.657224 14908 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:13:12.658823 14894 server_base.cc:1034] running on GCE node
I20260807 15:13:12.659049 14894 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:13:12.659379 14894 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:13:12.662680 14894 hybrid_clock.cc:648] HybridClock initialized: now 1786115592662657 us; error 33 us; skew 500 ppm
Aug 07 15:13:12 dist-test-slave-42z9 krb5kdc[13835](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115592, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.194.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:13:12.666772 14894 init.cc:377] Logged in from keytab as kudu/127.29.194.2@KRBTEST.COM (short username kudu)
I20260807 15:13:12.667397 14894 webserver.cc:460] Webserver started at http://127.29.194.2:41489/ using document root <none> and password file <none>
I20260807 15:13:12.667755 14894 fs_manager.cc:359] Metadata directory not provided
I20260807 15:13:12.667820 14894 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:13:12.667979 14894 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 15:13:12 dist-test-slave-42z9 krb5kdc[13835](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115592, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.194.2@KRBTEST.COM for HTTP/127.29.194.20@KRBTEST.COM
I20260807 15:13:12.688663 14894 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115480851507-30472-0/minicluster-data/ts-1/data/instance:
uuid: "4b3abc4963424d8baf9c8044f6e0cb35"
format_stamp: "Formatted at 2026-08-07 15:13:12 on dist-test-slave-42z9"
server_key: "367be6ba5bd721702baf3ce1b0b394c8"
server_key_iv: "73f01e87faac4fa131c688cb33e980ff"
server_key_version: "kuduclusterkey@0"
I20260807 15:13:12.689188 14894 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115480851507-30472-0/minicluster-data/ts-1/wal/instance:
uuid: "4b3abc4963424d8baf9c8044f6e0cb35"
format_stamp: "Formatted at 2026-08-07 15:13:12 on dist-test-slave-42z9"
server_key: "367be6ba5bd721702baf3ce1b0b394c8"
server_key_iv: "73f01e87faac4fa131c688cb33e980ff"
server_key_version: "kuduclusterkey@0"
I20260807 15:13:12.692327 14894 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260807 15:13:12.706125 14920 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:13:12.706290 14894 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:13:12.706378 14894 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115480851507-30472-0/minicluster-data/ts-1/data,/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115480851507-30472-0/minicluster-data/ts-1/wal
uuid: "4b3abc4963424d8baf9c8044f6e0cb35"
format_stamp: "Formatted at 2026-08-07 15:13:12 on dist-test-slave-42z9"
server_key: "367be6ba5bd721702baf3ce1b0b394c8"
server_key_iv: "73f01e87faac4fa131c688cb33e980ff"
server_key_version: "kuduclusterkey@0"
I20260807 15:13:12.706472 14894 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115480851507-30472-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115480851507-30472-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115480851507-30472-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:13:12.717809 14894 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:13:12.720192 14894 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:13:12.720350 14894 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:13:12.720599 14894 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:13:12.720959 14894 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:13:12.720999 14894 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:13:12.721030 14894 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:13:12.721081 14894 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:13:12.743445 14894 rpc_server.cc:307] RPC server started. Bound to: 127.29.194.2:36633
I20260807 15:13:12.744084 14894 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115480851507-30472-0/minicluster-data/ts-1/data/info.pb
I20260807 15:13:12.744920 15085 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.194.2:36633 every 8 connection(s)
I20260807 15:13:12.747488 30472 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7fZSIL/build/release/bin/kudu as pid 14894
I20260807 15:13:12.747599 30472 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115480851507-30472-0/minicluster-data/ts-1/wal/instance
I20260807 15:13:12.747804 30472 external_mini_cluster.cc:1428] Setting key 1c51cc9071fd0b5a018516cb9a99bee2
Aug 07 15:13:12 dist-test-slave-42z9 krb5kdc[13835](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115592, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.194.2@KRBTEST.COM for kudu/127.29.194.62@KRBTEST.COM
I20260807 15:13:12.761598 15091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:13:12.746194 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:37499 (local address 127.29.194.2:36467)
0807 15:13:12.746465 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0807 15:13:12.746509 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0807 15:13:12.747003 (+ 494us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:13:12.747335 (+ 332us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:13:12.747340 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:13:12.747799 (+ 459us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:13:12.748292 (+ 493us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:13:12.748299 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:13:12.749173 (+ 874us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:13:12.749174 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:13:12.750534 (+ 1360us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:13:12.750538 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:13:12.751093 (+ 555us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:13:12.751094 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:13:12.751247 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:13:12.751893 (+ 646us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:13:12.751904 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:13:12.757562 (+ 5658us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:13:12.759637 (+ 2075us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:13:12.759650 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:13:12.759653 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:13:12.760037 (+ 384us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:13:12.760289 (+ 252us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:13:12.760290 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:13:12.760290 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:13:12.760450 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:13:12.760776 (+ 326us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:13:12.760784 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:13:12.760994 (+ 210us) client_negotiation.cc:769] Sending connection context
0807 15:13:12.761116 (+ 122us) client_negotiation.cc:240] Negotiation successful
0807 15:13:12.761311 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":227,"thread_start_us":173,"threads_started":1}
I20260807 15:13:12.762351 14887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:13:12.746190 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.194.2:36467 (local address 127.29.194.62:37499)
0807 15:13:12.746238 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0807 15:13:12.746241 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:13:12.746603 (+ 362us) server_negotiation.cc:407] Connection header received
0807 15:13:12.747099 (+ 496us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:13:12.747100 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:13:12.747119 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:13:12.747185 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:13:12.748394 (+ 1209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:13:12.749059 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:13:12.750643 (+ 1584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:13:12.750950 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:13:12.750994 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:13:12.757723 (+ 6729us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:13:12.757725 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:13:12.757726 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:13:12.757736 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:13:12.759520 (+ 1784us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:13:12.760140 (+ 620us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:13:12.760142 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:13:12.760143 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:13:12.760219 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:13:12.760527 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:13:12.760529 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:13:12.760529 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:13:12.760687 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:13:12.760747 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0807 15:13:12.762159 (+ 1412us) server_negotiation.cc:299] Negotiation successful
0807 15:13:12.762268 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260807 15:13:12.762765 15086 heartbeater.cc:347] Connected to a master server at 127.29.194.62:37499
I20260807 15:13:12.762887 15086 heartbeater.cc:464] Registering TS with master...
I20260807 15:13:12.763105 15086 heartbeater.cc:511] Master 127.29.194.62:37499 requested a full tablet report, sending...
I20260807 15:13:12.763501 14565 ts_manager.cc:194] Registered new tserver with Master: 4b3abc4963424d8baf9c8044f6e0cb35 (127.29.194.2:36633)
I20260807 15:13:12.764007 14565 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.194.2@KRBTEST.COM'} at 127.29.194.2:36467
WARNING: no policy specified for kudu/127.29.194.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.194.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.194.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.3.keytab.
Entry for principal kudu/127.29.194.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.3.keytab.
I20260807 15:13:12.796384 30472 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7fZSIL/build/release/bin/kudu
/tmp/dist-test-task7fZSIL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115480851507-30472-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115480851507-30472-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115480851507-30472-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115480851507-30472-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.29.194.20:60841/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.194.3:0
--local_ip_for_outbound_sockets=127.29.194.3
--webserver_interface=127.29.194.3
--webserver_port=0
--tserver_master_addrs=127.29.194.62:37499
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.3.keytab
--principal=kudu/127.29.194.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.194.20:44037
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115480851507-30472-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115480851507-30472-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115480851507-30472-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:13:12.941488 15097 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:13:12.941927 15097 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:13:12.942000 15097 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:13:12.943955 15097 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:13:12.944027 15097 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:13:12.944085 15097 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.194.3
I20260807 15:13:12.946710 15097 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.194.20:44037
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115480851507-30472-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115480851507-30472-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.29.194.20:60841/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.3.keytab
--principal=kudu/127.29.194.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.194.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115480851507-30472-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.194.3
--webserver_port=0
--tserver_master_addrs=127.29.194.62:37499
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15097
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.194.3
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115480851507-30472-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:13:12.947405 15097 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:13:12.947841 15097 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:13:12.956278 15104 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:13:12.956355 15105 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:13:12.956745 15107 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:13:12.957947 15097 server_base.cc:1034] running on GCE node
I20260807 15:13:12.958310 15097 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:13:12.958609 15097 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:13:12.959785 15097 hybrid_clock.cc:648] HybridClock initialized: now 1786115592959758 us; error 41 us; skew 500 ppm
Aug 07 15:13:12 dist-test-slave-42z9 krb5kdc[13835](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115592, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.194.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:13:12.963707 15097 init.cc:377] Logged in from keytab as kudu/127.29.194.3@KRBTEST.COM (short username kudu)
I20260807 15:13:12.964357 15097 webserver.cc:460] Webserver started at http://127.29.194.3:38801/ using document root <none> and password file <none>
I20260807 15:13:12.964648 15097 fs_manager.cc:359] Metadata directory not provided
I20260807 15:13:12.964730 15097 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:13:12.964881 15097 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 15:13:12 dist-test-slave-42z9 krb5kdc[13835](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115592, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.194.3@KRBTEST.COM for HTTP/127.29.194.20@KRBTEST.COM
I20260807 15:13:12.983327 15097 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115480851507-30472-0/minicluster-data/ts-2/data/instance:
uuid: "49f32a76e00a408abd5a6e3029f550b8"
format_stamp: "Formatted at 2026-08-07 15:13:12 on dist-test-slave-42z9"
server_key: "e4b2ca25474e88a0ee76ba9148c68aeb"
server_key_iv: "2b0dc5509acd2a10a2f9c5a33133cc58"
server_key_version: "kuduclusterkey@0"
I20260807 15:13:12.983880 15097 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115480851507-30472-0/minicluster-data/ts-2/wal/instance:
uuid: "49f32a76e00a408abd5a6e3029f550b8"
format_stamp: "Formatted at 2026-08-07 15:13:12 on dist-test-slave-42z9"
server_key: "e4b2ca25474e88a0ee76ba9148c68aeb"
server_key_iv: "2b0dc5509acd2a10a2f9c5a33133cc58"
server_key_version: "kuduclusterkey@0"
I20260807 15:13:12.985904 15097 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.004s
I20260807 15:13:13.017343 15117 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:13:13.017697 15097 fs_manager.cc:725] Time spent opening block manager: real 0.011s user 0.001s sys 0.002s
I20260807 15:13:13.017834 15097 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115480851507-30472-0/minicluster-data/ts-2/data,/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115480851507-30472-0/minicluster-data/ts-2/wal
uuid: "49f32a76e00a408abd5a6e3029f550b8"
format_stamp: "Formatted at 2026-08-07 15:13:12 on dist-test-slave-42z9"
server_key: "e4b2ca25474e88a0ee76ba9148c68aeb"
server_key_iv: "2b0dc5509acd2a10a2f9c5a33133cc58"
server_key_version: "kuduclusterkey@0"
I20260807 15:13:13.017957 15097 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115480851507-30472-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115480851507-30472-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115480851507-30472-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:13:13.040933 15097 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:13:13.042964 15097 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:13:13.043107 15097 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:13:13.043385 15097 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:13:13.043756 15097 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:13:13.043800 15097 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:13:13.043831 15097 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:13:13.043870 15097 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:13:13.051373 15097 rpc_server.cc:307] RPC server started. Bound to: 127.29.194.3:33355
I20260807 15:13:13.051846 15097 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115480851507-30472-0/minicluster-data/ts-2/data/info.pb
I20260807 15:13:13.053848 15283 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.194.3:33355 every 8 connection(s)
I20260807 15:13:13.057020 30472 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7fZSIL/build/release/bin/kudu as pid 15097
I20260807 15:13:13.057191 30472 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115480851507-30472-0/minicluster-data/ts-2/wal/instance
I20260807 15:13:13.057487 30472 external_mini_cluster.cc:1428] Setting key ce98e00f6d64a28ac45c90bb62eca0c1
Aug 07 15:13:13 dist-test-slave-42z9 krb5kdc[13835](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115592, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.194.3@KRBTEST.COM for kudu/127.29.194.62@KRBTEST.COM
I20260807 15:13:13.066924 14887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:13:13.055209 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.194.3:35747 (local address 127.29.194.62:37499)
0807 15:13:13.055280 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0807 15:13:13.055283 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:13:13.057783 (+ 2500us) server_negotiation.cc:407] Connection header received
0807 15:13:13.058908 (+ 1125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:13:13.058909 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:13:13.058924 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:13:13.058999 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:13:13.059939 (+ 940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:13:13.060442 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:13:13.061499 (+ 1057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:13:13.061722 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:13:13.061780 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:13:13.063901 (+ 2121us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:13:13.063902 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:13:13.063902 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:13:13.063909 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:13:13.065250 (+ 1341us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:13:13.065783 (+ 533us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:13:13.065784 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:13:13.065785 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:13:13.065850 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:13:13.066154 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:13:13.066155 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:13:13.066155 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:13:13.066292 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:13:13.066343 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0807 15:13:13.066725 (+ 382us) server_negotiation.cc:299] Negotiation successful
0807 15:13:13.066826 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260807 15:13:13.067865 15287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:13:13.055200 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:37499 (local address 127.29.194.3:35747)
0807 15:13:13.057512 (+ 2312us) negotiation.cc:107] Waiting for socket to connect
0807 15:13:13.057611 (+ 99us) client_negotiation.cc:174] Beginning negotiation
0807 15:13:13.058793 (+ 1182us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:13:13.059301 (+ 508us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:13:13.059309 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:13:13.059564 (+ 255us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:13:13.059792 (+ 228us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:13:13.059797 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:13:13.060543 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:13:13.060544 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:13:13.061382 (+ 838us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:13:13.061385 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:13:13.061860 (+ 475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:13:13.061861 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:13:13.061974 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:13:13.062520 (+ 546us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:13:13.062530 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:13:13.063805 (+ 1275us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:13:13.065346 (+ 1541us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:13:13.065353 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:13:13.065356 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:13:13.065683 (+ 327us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:13:13.065924 (+ 241us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:13:13.065925 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:13:13.065925 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:13:13.066075 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:13:13.066378 (+ 303us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:13:13.066384 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:13:13.066555 (+ 171us) client_negotiation.cc:769] Sending connection context
0807 15:13:13.067426 (+ 871us) client_negotiation.cc:240] Negotiation successful
0807 15:13:13.067615 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2179,"thread_start_us":105,"threads_started":1}
I20260807 15:13:13.068615 15284 heartbeater.cc:347] Connected to a master server at 127.29.194.62:37499
I20260807 15:13:13.068728 15284 heartbeater.cc:464] Registering TS with master...
I20260807 15:13:13.068939 15284 heartbeater.cc:511] Master 127.29.194.62:37499 requested a full tablet report, sending...
I20260807 15:13:13.069387 14565 ts_manager.cc:194] Registered new tserver with Master: 49f32a76e00a408abd5a6e3029f550b8 (127.29.194.3:33355)
I20260807 15:13:13.069959 14565 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.194.3@KRBTEST.COM'} at 127.29.194.3:35747
I20260807 15:13:13.070518 30472 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 15:13:13.094743 14887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:13:13.085934 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50754 (local address 127.29.194.62:37499)
0807 15:13:13.085994 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0807 15:13:13.085996 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:13:13.086496 (+ 500us) server_negotiation.cc:407] Connection header received
0807 15:13:13.086611 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:13:13.086613 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:13:13.086626 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:13:13.086657 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:13:13.087667 (+ 1010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:13:13.088466 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:13:13.089557 (+ 1091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:13:13.089858 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:13:13.089904 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:13:13.091662 (+ 1758us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:13:13.091664 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:13:13.091664 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:13:13.091672 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:13:13.093319 (+ 1647us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:13:13.093785 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:13:13.093786 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:13:13.093787 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:13:13.093850 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:13:13.094083 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:13:13.094084 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:13:13.094085 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:13:13.094225 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:13:13.094307 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0807 15:13:13.094545 (+ 238us) server_negotiation.cc:299] Negotiation successful
0807 15:13:13.094640 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260807 15:13:13.097144 14565 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50754:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 15:13:13.099771 14565 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 15:13:13.123917 15312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:13:13.113781 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.1:44229 (local address 127.0.0.1:38592)
0807 15:13:13.116905 (+ 3124us) negotiation.cc:107] Waiting for socket to connect
0807 15:13:13.116913 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0807 15:13:13.116964 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:13:13.117660 (+ 696us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:13:13.117664 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:13:13.117666 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:13:13.117754 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:13:13.117756 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:13:13.121100 (+ 3344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:13:13.121101 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:13:13.122466 (+ 1365us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:13:13.122470 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:13:13.123581 (+ 1111us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:13:13.123582 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:13:13.123670 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:13:13.123686 (+ 16us) client_negotiation.cc:769] Sending connection context
0807 15:13:13.123758 (+ 72us) client_negotiation.cc:240] Negotiation successful
0807 15:13:13.123821 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3108,"thread_start_us":129,"threads_started":1}
I20260807 15:13:13.124459 15313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:13:13.114100 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38592 (local address 127.29.194.1:44229)
0807 15:13:13.116844 (+ 2744us) server_negotiation.cc:206] Beginning negotiation
0807 15:13:13.116848 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:13:13.117017 (+ 169us) server_negotiation.cc:407] Connection header received
0807 15:13:13.117068 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:13:13.117070 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:13:13.117087 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:13:13.117157 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:13:13.118021 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:13:13.119425 (+ 1404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:13:13.122582 (+ 3157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:13:13.123394 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:13:13.123473 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:13:13.123540 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0807 15:13:13.124331 (+ 791us) server_negotiation.cc:299] Negotiation successful
0807 15:13:13.124387 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2704,"thread_start_us":67,"threads_started":1}
I20260807 15:13:13.125386 14777 tablet_service.cc:1467] Processing CreateTablet for tablet 472327e331c24d238554e40d2d4444aa (DEFAULT_TABLE table=test-table [id=2b034955e4a741e0b412be2de1273c9b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:13:13.125831 14777 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 472327e331c24d238554e40d2d4444aa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:13:13.135543 15307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:13:13.113109 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45878 (local address 127.29.194.3:33355)
0807 15:13:13.116396 (+ 3287us) server_negotiation.cc:206] Beginning negotiation
0807 15:13:13.116399 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:13:13.116410 (+ 11us) server_negotiation.cc:407] Connection header received
0807 15:13:13.116452 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:13:13.116454 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:13:13.116472 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:13:13.116550 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:13:13.119821 (+ 3271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:13:13.120976 (+ 1155us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:13:13.131806 (+ 10830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:13:13.132621 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:13:13.132676 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:13:13.132761 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0807 15:13:13.135286 (+ 2525us) server_negotiation.cc:299] Negotiation successful
0807 15:13:13.135350 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3249,"thread_start_us":147,"threads_started":1}
I20260807 15:13:13.136076 15304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:13:13.112481 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.3:33355 (local address 127.0.0.1:45878)
0807 15:13:13.115987 (+ 3506us) negotiation.cc:107] Waiting for socket to connect
0807 15:13:13.116057 (+ 70us) client_negotiation.cc:174] Beginning negotiation
0807 15:13:13.116140 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:13:13.119530 (+ 3390us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:13:13.119531 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:13:13.119534 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:13:13.119716 (+ 182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:13:13.119719 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:13:13.121903 (+ 2184us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:13:13.121905 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:13:13.131273 (+ 9368us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:13:13.131279 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:13:13.134157 (+ 2878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:13:13.134157 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:13:13.134231 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:13:13.134236 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 15:13:13.135919 (+ 1683us) client_negotiation.cc:240] Negotiation successful
0807 15:13:13.135986 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3464,"thread_start_us":106,"threads_started":1}
I20260807 15:13:13.137161 15186 tablet_service.cc:1467] Processing CreateTablet for tablet 472327e331c24d238554e40d2d4444aa (DEFAULT_TABLE table=test-table [id=2b034955e4a741e0b412be2de1273c9b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:13:13.137687 15186 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 472327e331c24d238554e40d2d4444aa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:13:13.143851 15305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:13:13.112742 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.2:36633 (local address 127.0.0.1:43988)
0807 15:13:13.116664 (+ 3922us) negotiation.cc:107] Waiting for socket to connect
0807 15:13:13.116675 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 15:13:13.116728 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:13:13.117839 (+ 1111us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:13:13.117840 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:13:13.117842 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:13:13.117931 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:13:13.117933 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:13:13.132834 (+ 14901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:13:13.132836 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:13:13.134053 (+ 1217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:13:13.134059 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:13:13.143542 (+ 9483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:13:13.143543 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:13:13.143630 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:13:13.143637 (+ 7us) client_negotiation.cc:769] Sending connection context
0807 15:13:13.143665 (+ 28us) client_negotiation.cc:240] Negotiation successful
0807 15:13:13.143719 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3903,"thread_start_us":66,"threads_started":1}
I20260807 15:13:13.144106 15314 tablet_bootstrap.cc:492] T 472327e331c24d238554e40d2d4444aa P 2b2738d42cfd4be5865540ff0ec11dc9: Bootstrap starting.
I20260807 15:13:13.144603 15311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:13:13.113348 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43988 (local address 127.29.194.2:36633)
0807 15:13:13.116298 (+ 2950us) server_negotiation.cc:206] Beginning negotiation
0807 15:13:13.116303 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:13:13.117227 (+ 924us) server_negotiation.cc:407] Connection header received
0807 15:13:13.117273 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:13:13.117275 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:13:13.117290 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:13:13.117358 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:13:13.122433 (+ 5075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:13:13.131673 (+ 9240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:13:13.134425 (+ 2752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:13:13.143417 (+ 8992us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:13:13.144285 (+ 868us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:13:13.144393 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0807 15:13:13.144430 (+ 37us) server_negotiation.cc:299] Negotiation successful
0807 15:13:13.144479 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2910,"thread_start_us":277,"threads_started":1}
I20260807 15:13:13.145026 15314 tablet_bootstrap.cc:654] T 472327e331c24d238554e40d2d4444aa P 2b2738d42cfd4be5865540ff0ec11dc9: Neither blocks nor log segments found. Creating new log.
I20260807 15:13:13.145535 14972 tablet_service.cc:1467] Processing CreateTablet for tablet 472327e331c24d238554e40d2d4444aa (DEFAULT_TABLE table=test-table [id=2b034955e4a741e0b412be2de1273c9b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:13:13.145769 15314 log.cc:824] T 472327e331c24d238554e40d2d4444aa P 2b2738d42cfd4be5865540ff0ec11dc9: Log is configured to *not* fsync() on all Append() calls
I20260807 15:13:13.145943 14972 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 472327e331c24d238554e40d2d4444aa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:13:13.146353 15315 tablet_bootstrap.cc:492] T 472327e331c24d238554e40d2d4444aa P 49f32a76e00a408abd5a6e3029f550b8: Bootstrap starting.
I20260807 15:13:13.147902 15315 tablet_bootstrap.cc:654] T 472327e331c24d238554e40d2d4444aa P 49f32a76e00a408abd5a6e3029f550b8: Neither blocks nor log segments found. Creating new log.
I20260807 15:13:13.148346 15315 log.cc:824] T 472327e331c24d238554e40d2d4444aa P 49f32a76e00a408abd5a6e3029f550b8: Log is configured to *not* fsync() on all Append() calls
I20260807 15:13:13.153784 15318 tablet_bootstrap.cc:492] T 472327e331c24d238554e40d2d4444aa P 4b3abc4963424d8baf9c8044f6e0cb35: Bootstrap starting.
I20260807 15:13:13.156988 15314 tablet_bootstrap.cc:492] T 472327e331c24d238554e40d2d4444aa P 2b2738d42cfd4be5865540ff0ec11dc9: No bootstrap required, opened a new log
I20260807 15:13:13.157092 15314 ts_tablet_manager.cc:1397] T 472327e331c24d238554e40d2d4444aa P 2b2738d42cfd4be5865540ff0ec11dc9: Time spent bootstrapping tablet: real 0.013s user 0.000s sys 0.002s
I20260807 15:13:13.158740 15318 tablet_bootstrap.cc:654] T 472327e331c24d238554e40d2d4444aa P 4b3abc4963424d8baf9c8044f6e0cb35: Neither blocks nor log segments found. Creating new log.
I20260807 15:13:13.159802 15318 log.cc:824] T 472327e331c24d238554e40d2d4444aa P 4b3abc4963424d8baf9c8044f6e0cb35: Log is configured to *not* fsync() on all Append() calls
I20260807 15:13:13.160954 15318 tablet_bootstrap.cc:492] T 472327e331c24d238554e40d2d4444aa P 4b3abc4963424d8baf9c8044f6e0cb35: No bootstrap required, opened a new log
I20260807 15:13:13.161041 15318 ts_tablet_manager.cc:1397] T 472327e331c24d238554e40d2d4444aa P 4b3abc4963424d8baf9c8044f6e0cb35: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20260807 15:13:13.161306 15314 raft_consensus.cc:348] T 472327e331c24d238554e40d2d4444aa P 2b2738d42cfd4be5865540ff0ec11dc9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f32a76e00a408abd5a6e3029f550b8" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 33355 } } peers { permanent_uuid: "2b2738d42cfd4be5865540ff0ec11dc9" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 44229 } } peers { permanent_uuid: "4b3abc4963424d8baf9c8044f6e0cb35" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 36633 } }
I20260807 15:13:13.161538 15314 raft_consensus.cc:374] T 472327e331c24d238554e40d2d4444aa P 2b2738d42cfd4be5865540ff0ec11dc9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:13:13.161583 15314 raft_consensus.cc:729] T 472327e331c24d238554e40d2d4444aa P 2b2738d42cfd4be5865540ff0ec11dc9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b2738d42cfd4be5865540ff0ec11dc9, State: Initialized, Role: FOLLOWER
I20260807 15:13:13.161737 15314 consensus_queue.cc:260] T 472327e331c24d238554e40d2d4444aa P 2b2738d42cfd4be5865540ff0ec11dc9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f32a76e00a408abd5a6e3029f550b8" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 33355 } } peers { permanent_uuid: "2b2738d42cfd4be5865540ff0ec11dc9" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 44229 } } peers { permanent_uuid: "4b3abc4963424d8baf9c8044f6e0cb35" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 36633 } }
I20260807 15:13:13.162057 15314 ts_tablet_manager.cc:1428] T 472327e331c24d238554e40d2d4444aa P 2b2738d42cfd4be5865540ff0ec11dc9: Time spent starting tablet: real 0.005s user 0.002s sys 0.003s
I20260807 15:13:13.162184 14882 heartbeater.cc:503] Master 127.29.194.62:37499 was elected leader, sending a full tablet report...
I20260807 15:13:13.163614 15318 raft_consensus.cc:348] T 472327e331c24d238554e40d2d4444aa P 4b3abc4963424d8baf9c8044f6e0cb35 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f32a76e00a408abd5a6e3029f550b8" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 33355 } } peers { permanent_uuid: "2b2738d42cfd4be5865540ff0ec11dc9" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 44229 } } peers { permanent_uuid: "4b3abc4963424d8baf9c8044f6e0cb35" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 36633 } }
I20260807 15:13:13.163770 15318 raft_consensus.cc:374] T 472327e331c24d238554e40d2d4444aa P 4b3abc4963424d8baf9c8044f6e0cb35 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:13:13.163808 15318 raft_consensus.cc:729] T 472327e331c24d238554e40d2d4444aa P 4b3abc4963424d8baf9c8044f6e0cb35 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b3abc4963424d8baf9c8044f6e0cb35, State: Initialized, Role: FOLLOWER
I20260807 15:13:13.164904 15318 consensus_queue.cc:260] T 472327e331c24d238554e40d2d4444aa P 4b3abc4963424d8baf9c8044f6e0cb35 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f32a76e00a408abd5a6e3029f550b8" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 33355 } } peers { permanent_uuid: "2b2738d42cfd4be5865540ff0ec11dc9" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 44229 } } peers { permanent_uuid: "4b3abc4963424d8baf9c8044f6e0cb35" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 36633 } }
I20260807 15:13:13.165302 15318 ts_tablet_manager.cc:1428] T 472327e331c24d238554e40d2d4444aa P 4b3abc4963424d8baf9c8044f6e0cb35: Time spent starting tablet: real 0.004s user 0.000s sys 0.003s
I20260807 15:13:13.165500 15086 heartbeater.cc:503] Master 127.29.194.62:37499 was elected leader, sending a full tablet report...
I20260807 15:13:13.166204 15315 tablet_bootstrap.cc:492] T 472327e331c24d238554e40d2d4444aa P 49f32a76e00a408abd5a6e3029f550b8: No bootstrap required, opened a new log
I20260807 15:13:13.166291 15315 ts_tablet_manager.cc:1397] T 472327e331c24d238554e40d2d4444aa P 49f32a76e00a408abd5a6e3029f550b8: Time spent bootstrapping tablet: real 0.020s user 0.003s sys 0.000s
I20260807 15:13:13.168736 15315 raft_consensus.cc:348] T 472327e331c24d238554e40d2d4444aa P 49f32a76e00a408abd5a6e3029f550b8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f32a76e00a408abd5a6e3029f550b8" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 33355 } } peers { permanent_uuid: "2b2738d42cfd4be5865540ff0ec11dc9" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 44229 } } peers { permanent_uuid: "4b3abc4963424d8baf9c8044f6e0cb35" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 36633 } }
I20260807 15:13:13.169056 15315 raft_consensus.cc:374] T 472327e331c24d238554e40d2d4444aa P 49f32a76e00a408abd5a6e3029f550b8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:13:13.169163 15315 raft_consensus.cc:729] T 472327e331c24d238554e40d2d4444aa P 49f32a76e00a408abd5a6e3029f550b8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49f32a76e00a408abd5a6e3029f550b8, State: Initialized, Role: FOLLOWER
I20260807 15:13:13.169384 15315 consensus_queue.cc:260] T 472327e331c24d238554e40d2d4444aa P 49f32a76e00a408abd5a6e3029f550b8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f32a76e00a408abd5a6e3029f550b8" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 33355 } } peers { permanent_uuid: "2b2738d42cfd4be5865540ff0ec11dc9" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 44229 } } peers { permanent_uuid: "4b3abc4963424d8baf9c8044f6e0cb35" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 36633 } }
I20260807 15:13:13.169826 15315 ts_tablet_manager.cc:1428] T 472327e331c24d238554e40d2d4444aa P 49f32a76e00a408abd5a6e3029f550b8: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260807 15:13:13.170017 15284 heartbeater.cc:503] Master 127.29.194.62:37499 was elected leader, sending a full tablet report...
I20260807 15:13:13.214095 15320 raft_consensus.cc:482] T 472327e331c24d238554e40d2d4444aa P 2b2738d42cfd4be5865540ff0ec11dc9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 15:13:13.214354 15320 raft_consensus.cc:504] T 472327e331c24d238554e40d2d4444aa P 2b2738d42cfd4be5865540ff0ec11dc9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f32a76e00a408abd5a6e3029f550b8" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 33355 } } peers { permanent_uuid: "2b2738d42cfd4be5865540ff0ec11dc9" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 44229 } } peers { permanent_uuid: "4b3abc4963424d8baf9c8044f6e0cb35" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 36633 } }
I20260807 15:13:13.215976 15320 leader_election.cc:288] T 472327e331c24d238554e40d2d4444aa P 2b2738d42cfd4be5865540ff0ec11dc9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 49f32a76e00a408abd5a6e3029f550b8 (127.29.194.3:33355), 4b3abc4963424d8baf9c8044f6e0cb35 (127.29.194.2:36633)
W20260807 15:13:13.217478 14883 tablet.cc:2369] T 472327e331c24d238554e40d2d4444aa P 2b2738d42cfd4be5865540ff0ec11dc9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 15:13:13.220710 15307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:13:13.215506 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.194.1:51281 (local address 127.29.194.3:33355)
0807 15:13:13.215556 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0807 15:13:13.215558 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:13:13.216867 (+ 1309us) server_negotiation.cc:407] Connection header received
0807 15:13:13.216910 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:13:13.216912 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:13:13.216925 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:13:13.217017 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:13:13.217473 (+ 456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:13:13.218204 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:13:13.219807 (+ 1603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:13:13.220329 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:13:13.220380 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:13:13.220388 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0807 15:13:13.220596 (+ 208us) server_negotiation.cc:299] Negotiation successful
0807 15:13:13.220641 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260807 15:13:13.221184 15325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:13:13.214982 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.3:33355 (local address 127.29.194.1:51281)
0807 15:13:13.216756 (+ 1774us) negotiation.cc:107] Waiting for socket to connect
0807 15:13:13.216769 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 15:13:13.216848 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:13:13.217081 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:13:13.217082 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:13:13.217084 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:13:13.217344 (+ 260us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:13:13.217347 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:13:13.218337 (+ 990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:13:13.218338 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:13:13.219699 (+ 1361us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:13:13.219703 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:13:13.220450 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:13:13.220451 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:13:13.220523 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:13:13.220527 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 15:13:13.221048 (+ 521us) client_negotiation.cc:240] Negotiation successful
0807 15:13:13.221102 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1741,"thread_start_us":145,"threads_started":1}
I20260807 15:13:13.221740 15189 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "472327e331c24d238554e40d2d4444aa" candidate_uuid: "2b2738d42cfd4be5865540ff0ec11dc9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49f32a76e00a408abd5a6e3029f550b8" is_pre_election: true
I20260807 15:13:13.221923 15189 raft_consensus.cc:2454] T 472327e331c24d238554e40d2d4444aa P 49f32a76e00a408abd5a6e3029f550b8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2b2738d42cfd4be5865540ff0ec11dc9 in term 0.
I20260807 15:13:13.222338 14712 leader_election.cc:302] T 472327e331c24d238554e40d2d4444aa P 2b2738d42cfd4be5865540ff0ec11dc9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2b2738d42cfd4be5865540ff0ec11dc9, 49f32a76e00a408abd5a6e3029f550b8; no voters:
I20260807 15:13:13.222476 15320 raft_consensus.cc:2781] T 472327e331c24d238554e40d2d4444aa P 2b2738d42cfd4be5865540ff0ec11dc9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 15:13:13.222514 15320 raft_consensus.cc:482] T 472327e331c24d238554e40d2d4444aa P 2b2738d42cfd4be5865540ff0ec11dc9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 15:13:13.222533 15320 raft_consensus.cc:3037] T 472327e331c24d238554e40d2d4444aa P 2b2738d42cfd4be5865540ff0ec11dc9 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:13:13.223474 15320 raft_consensus.cc:504] T 472327e331c24d238554e40d2d4444aa P 2b2738d42cfd4be5865540ff0ec11dc9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f32a76e00a408abd5a6e3029f550b8" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 33355 } } peers { permanent_uuid: "2b2738d42cfd4be5865540ff0ec11dc9" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 44229 } } peers { permanent_uuid: "4b3abc4963424d8baf9c8044f6e0cb35" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 36633 } }
I20260807 15:13:13.223616 15320 leader_election.cc:288] T 472327e331c24d238554e40d2d4444aa P 2b2738d42cfd4be5865540ff0ec11dc9 [CANDIDATE]: Term 1 election: Requested vote from peers 49f32a76e00a408abd5a6e3029f550b8 (127.29.194.3:33355), 4b3abc4963424d8baf9c8044f6e0cb35 (127.29.194.2:36633)
I20260807 15:13:13.224013 15189 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "472327e331c24d238554e40d2d4444aa" candidate_uuid: "2b2738d42cfd4be5865540ff0ec11dc9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49f32a76e00a408abd5a6e3029f550b8"
I20260807 15:13:13.224097 15189 raft_consensus.cc:3037] T 472327e331c24d238554e40d2d4444aa P 49f32a76e00a408abd5a6e3029f550b8 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:13:13.224936 15189 raft_consensus.cc:2454] T 472327e331c24d238554e40d2d4444aa P 49f32a76e00a408abd5a6e3029f550b8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2b2738d42cfd4be5865540ff0ec11dc9 in term 1.
I20260807 15:13:13.225214 14712 leader_election.cc:302] T 472327e331c24d238554e40d2d4444aa P 2b2738d42cfd4be5865540ff0ec11dc9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2b2738d42cfd4be5865540ff0ec11dc9, 49f32a76e00a408abd5a6e3029f550b8; no voters:
I20260807 15:13:13.225337 15320 raft_consensus.cc:2781] T 472327e331c24d238554e40d2d4444aa P 2b2738d42cfd4be5865540ff0ec11dc9 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:13:13.225406 15311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:13:13.215913 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.194.1:55831 (local address 127.29.194.2:36633)
0807 15:13:13.215956 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0807 15:13:13.215958 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:13:13.220050 (+ 4092us) server_negotiation.cc:407] Connection header received
0807 15:13:13.220156 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:13:13.220157 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:13:13.220167 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:13:13.220196 (+ 29us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:13:13.220471 (+ 275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:13:13.221207 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:13:13.223529 (+ 2322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:13:13.224114 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:13:13.224166 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:13:13.224176 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0807 15:13:13.225270 (+ 1094us) server_negotiation.cc:299] Negotiation successful
0807 15:13:13.225332 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260807 15:13:13.225550 15320 raft_consensus.cc:686] T 472327e331c24d238554e40d2d4444aa P 2b2738d42cfd4be5865540ff0ec11dc9 [term 1 LEADER]: Becoming Leader. State: Replica: 2b2738d42cfd4be5865540ff0ec11dc9, State: Running, Role: LEADER
I20260807 15:13:13.225648 15320 consensus_queue.cc:237] T 472327e331c24d238554e40d2d4444aa P 2b2738d42cfd4be5865540ff0ec11dc9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f32a76e00a408abd5a6e3029f550b8" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 33355 } } peers { permanent_uuid: "2b2738d42cfd4be5865540ff0ec11dc9" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 44229 } } peers { permanent_uuid: "4b3abc4963424d8baf9c8044f6e0cb35" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 36633 } }
I20260807 15:13:13.225706 15326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:13:13.215725 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.2:36633 (local address 127.29.194.1:55831)
0807 15:13:13.219953 (+ 4228us) negotiation.cc:107] Waiting for socket to connect
0807 15:13:13.219969 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 15:13:13.220086 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:13:13.220259 (+ 173us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:13:13.220259 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:13:13.220261 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:13:13.220388 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:13:13.220389 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:13:13.221320 (+ 931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:13:13.221321 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:13:13.222619 (+ 1298us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:13:13.222623 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:13:13.224241 (+ 1618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:13:13.224242 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:13:13.224322 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:13:13.224326 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 15:13:13.225551 (+ 1225us) client_negotiation.cc:240] Negotiation successful
0807 15:13:13.225607 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1535,"thread_start_us":90,"threads_started":1}
I20260807 15:13:13.226668 15012 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "472327e331c24d238554e40d2d4444aa" candidate_uuid: "2b2738d42cfd4be5865540ff0ec11dc9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b3abc4963424d8baf9c8044f6e0cb35" is_pre_election: true
I20260807 15:13:13.226822 15012 raft_consensus.cc:2454] T 472327e331c24d238554e40d2d4444aa P 4b3abc4963424d8baf9c8044f6e0cb35 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2b2738d42cfd4be5865540ff0ec11dc9 in term 0.
I20260807 15:13:13.226737 14564 catalog_manager.cc:5495] T 472327e331c24d238554e40d2d4444aa P 2b2738d42cfd4be5865540ff0ec11dc9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2b2738d42cfd4be5865540ff0ec11dc9 (127.29.194.1). New cstate: current_term: 1 leader_uuid: "2b2738d42cfd4be5865540ff0ec11dc9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49f32a76e00a408abd5a6e3029f550b8" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 33355 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2b2738d42cfd4be5865540ff0ec11dc9" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 44229 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4b3abc4963424d8baf9c8044f6e0cb35" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 36633 } health_report { overall_health: UNKNOWN } } }
I20260807 15:13:13.226944 15011 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "472327e331c24d238554e40d2d4444aa" candidate_uuid: "2b2738d42cfd4be5865540ff0ec11dc9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b3abc4963424d8baf9c8044f6e0cb35"
I20260807 15:13:13.227002 15011 raft_consensus.cc:3037] T 472327e331c24d238554e40d2d4444aa P 4b3abc4963424d8baf9c8044f6e0cb35 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:13:13.228399 15011 raft_consensus.cc:2454] T 472327e331c24d238554e40d2d4444aa P 4b3abc4963424d8baf9c8044f6e0cb35 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2b2738d42cfd4be5865540ff0ec11dc9 in term 1.
W20260807 15:13:13.246369 15089 tablet.cc:2369] T 472327e331c24d238554e40d2d4444aa P 4b3abc4963424d8baf9c8044f6e0cb35: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 15:13:13.274766 15313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:13:13.271201 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38608 (local address 127.29.194.1:44229)
0807 15:13:13.271260 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0807 15:13:13.271263 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:13:13.271274 (+ 11us) server_negotiation.cc:407] Connection header received
0807 15:13:13.271326 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:13:13.271328 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:13:13.271356 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:13:13.271461 (+ 105us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 15:13:13.271614 (+ 153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:13:13.272393 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:13:13.273594 (+ 1201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:13:13.273904 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:13:13.273970 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:13:13.274174 (+ 204us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 15:13:13.274321 (+ 147us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 15:13:13.274391 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0807 15:13:13.274626 (+ 235us) server_negotiation.cc:299] Negotiation successful
0807 15:13:13.274653 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
Aug 07 15:13:13 dist-test-slave-42z9 krb5kdc[13835](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115592, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.194.62@KRBTEST.COM for HTTP/127.29.194.20@KRBTEST.COM
I20260807 15:13:13.284291 15011 raft_consensus.cc:1261] T 472327e331c24d238554e40d2d4444aa P 4b3abc4963424d8baf9c8044f6e0cb35 [term 1 FOLLOWER]: Refusing update from remote peer 2b2738d42cfd4be5865540ff0ec11dc9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:13:13.284801 15327 consensus_queue.cc:1035] T 472327e331c24d238554e40d2d4444aa P 2b2738d42cfd4be5865540ff0ec11dc9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b3abc4963424d8baf9c8044f6e0cb35" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 36633 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:13:13.287150 15189 raft_consensus.cc:1261] T 472327e331c24d238554e40d2d4444aa P 49f32a76e00a408abd5a6e3029f550b8 [term 1 FOLLOWER]: Refusing update from remote peer 2b2738d42cfd4be5865540ff0ec11dc9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:13:13.287743 15327 consensus_queue.cc:1035] T 472327e331c24d238554e40d2d4444aa P 2b2738d42cfd4be5865540ff0ec11dc9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49f32a76e00a408abd5a6e3029f550b8" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 33355 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:13:13.292371 15339 mvcc.cc:205] Tried to move back new op lower bound from 7315929470086692864 to 7315929469850624000. Current Snapshot: MvccSnapshot[applied={T|T < 7315929470086692864}]
I20260807 15:13:13.292729 15343 mvcc.cc:205] Tried to move back new op lower bound from 7315929470086692864 to 7315929469850624000. Current Snapshot: MvccSnapshot[applied={T|T < 7315929470086692864}]
I20260807 15:13:13.295822 15341 mvcc.cc:205] Tried to move back new op lower bound from 7315929470086692864 to 7315929469850624000. Current Snapshot: MvccSnapshot[applied={T|T < 7315929470086692864}]
I20260807 15:13:13.299098 14564 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50754:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 15:13:13.299259 14564 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50754:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 15:13:13.300187 14564 catalog_manager.cc:5782] T 00000000000000000000000000000000 P de41beabaa564ca294874d6874de509e: Sending DeleteTablet for 3 replicas of tablet 472327e331c24d238554e40d2d4444aa
I20260807 15:13:13.300647 15186 tablet_service.cc:1514] Processing DeleteTablet for tablet 472327e331c24d238554e40d2d4444aa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:13:13 UTC) from {username='kudu', principal='kudu/127.29.194.62@KRBTEST.COM'} at 127.0.0.1:45878
I20260807 15:13:13.300887 14972 tablet_service.cc:1514] Processing DeleteTablet for tablet 472327e331c24d238554e40d2d4444aa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:13:13 UTC) from {username='kudu', principal='kudu/127.29.194.62@KRBTEST.COM'} at 127.0.0.1:43988
I20260807 15:13:13.301352 14777 tablet_service.cc:1514] Processing DeleteTablet for tablet 472327e331c24d238554e40d2d4444aa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:13:13 UTC) from {username='kudu', principal='kudu/127.29.194.62@KRBTEST.COM'} at 127.0.0.1:38592
I20260807 15:13:13.302636 30472 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7fZSIL/build/release/bin/kudu with pid 14685
I20260807 15:13:13.317548 15352 tablet_replica.cc:330] T 472327e331c24d238554e40d2d4444aa P 49f32a76e00a408abd5a6e3029f550b8: stopping tablet replica
I20260807 15:13:13.317803 15353 tablet_replica.cc:330] T 472327e331c24d238554e40d2d4444aa P 4b3abc4963424d8baf9c8044f6e0cb35: stopping tablet replica
I20260807 15:13:13.318105 15353 raft_consensus.cc:2229] T 472327e331c24d238554e40d2d4444aa P 4b3abc4963424d8baf9c8044f6e0cb35 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 15:13:13.318287 15352 raft_consensus.cc:2229] T 472327e331c24d238554e40d2d4444aa P 49f32a76e00a408abd5a6e3029f550b8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 15:13:13.318502 15353 raft_consensus.cc:2258] T 472327e331c24d238554e40d2d4444aa P 4b3abc4963424d8baf9c8044f6e0cb35 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:13:13.318699 15352 raft_consensus.cc:2258] T 472327e331c24d238554e40d2d4444aa P 49f32a76e00a408abd5a6e3029f550b8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:13:13.319527 15352 ts_tablet_manager.cc:1905] T 472327e331c24d238554e40d2d4444aa P 49f32a76e00a408abd5a6e3029f550b8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:13:13.323274 15353 ts_tablet_manager.cc:1905] T 472327e331c24d238554e40d2d4444aa P 4b3abc4963424d8baf9c8044f6e0cb35: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:13:13.325219 15353 ts_tablet_manager.cc:1918] T 472327e331c24d238554e40d2d4444aa P 4b3abc4963424d8baf9c8044f6e0cb35: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 15:13:13.325420 15353 log.cc:1196] T 472327e331c24d238554e40d2d4444aa P 4b3abc4963424d8baf9c8044f6e0cb35: Deleting WAL directory at /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115480851507-30472-0/minicluster-data/ts-1/wal/wals/472327e331c24d238554e40d2d4444aa
I20260807 15:13:13.325615 15352 ts_tablet_manager.cc:1918] T 472327e331c24d238554e40d2d4444aa P 49f32a76e00a408abd5a6e3029f550b8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 15:13:13.325791 15353 ts_tablet_manager.cc:1939] T 472327e331c24d238554e40d2d4444aa P 4b3abc4963424d8baf9c8044f6e0cb35: Deleting consensus metadata
I20260807 15:13:13.325798 15352 log.cc:1196] T 472327e331c24d238554e40d2d4444aa P 49f32a76e00a408abd5a6e3029f550b8: Deleting WAL directory at /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115480851507-30472-0/minicluster-data/ts-2/wal/wals/472327e331c24d238554e40d2d4444aa
I20260807 15:13:13.326370 14537 catalog_manager.cc:4841] TS 4b3abc4963424d8baf9c8044f6e0cb35 (127.29.194.2:36633): tablet 472327e331c24d238554e40d2d4444aa (table test-table [id=2b034955e4a741e0b412be2de1273c9b]) successfully deleted
I20260807 15:13:13.326525 15352 ts_tablet_manager.cc:1939] T 472327e331c24d238554e40d2d4444aa P 49f32a76e00a408abd5a6e3029f550b8: Deleting consensus metadata
I20260807 15:13:13.327008 14542 catalog_manager.cc:4841] TS 49f32a76e00a408abd5a6e3029f550b8 (127.29.194.3:33355): tablet 472327e331c24d238554e40d2d4444aa (table test-table [id=2b034955e4a741e0b412be2de1273c9b]) successfully deleted
W20260807 15:13:13.328971 14540 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.29.194.1:44229 (error 108)
I20260807 15:13:13.329793 30472 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7fZSIL/build/release/bin/kudu with pid 14894
I20260807 15:13:13.349145 30472 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7fZSIL/build/release/bin/kudu with pid 15097
I20260807 15:13:13.350210 15305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:13:13.350024 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.1:44229 (local address 127.0.0.1:38610)
0807 15:13:13.350116 (+ 92us) negotiation.cc:107] Waiting for socket to connect
0807 15:13:13.350165 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.194.1:44229: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
W20260807 15:13:13.351691 14540 catalog_manager.cc:4584] TS 2b2738d42cfd4be5865540ff0ec11dc9 (127.29.194.1:44229): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 472327e331c24d238554e40d2d4444aa: Network error: Client connection negotiation failed: client connection to 127.29.194.1:44229: connect: Connection refused (error 111)
I20260807 15:13:13.361258 30472 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7fZSIL/build/release/bin/kudu with pid 14514
I20260807 15:13:13.373898 30472 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260807 15:13:13.721339 30472 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260807 15:13:13.721529 30472 mini_ranger.cc:66] Stopping Ranger...
I20260807 15:13:14.066845 30472 mini_ranger.cc:68] Stopped Ranger
2026-08-07 15:13:14.067 UTC [13887] LOG: received smart shutdown request
2026-08-07 15:13:14.071 UTC [13887] LOG: background worker "logical replication launcher" (PID 13897) exited with exit code 1
2026-08-07 15:13:14.072 UTC [13891] LOG: shutting down
2026-08-07 15:13:14.258 UTC [13887] LOG: database system is shut down
2026-08-07T15:13:14Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (51589 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115480851507-30472-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 15:13:14 dist-test-slave-42z9 krb5kdc[15367](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 15:13:14 dist-test-slave-42z9 krb5kdc[15367](info): set up 2 sockets
Aug 07 15:13:14 dist-test-slave-42z9 krb5kdc[15367](info): commencing operation
krb5kdc: starting...
W20260807 15:13:15.509316 30472 mini_kdc.cc:121] Time spent starting KDC: real 1.138s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115480851507-30472-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115480851507-30472-0/krb5kdc/test-user.keytab.
Aug 07 15:13:15 dist-test-slave-42z9 krb5kdc[15367](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115595, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T15:13:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T15:13:15Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.29.194.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.29.194.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.29.194.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115480851507-30472-0/krb5kdc/rangeradmin_127.29.194.20@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.29.194.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115480851507-30472-0/krb5kdc/rangeradmin_127.29.194.20@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.29.194.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.29.194.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.29.194.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115480851507-30472-0/krb5kdc/rangerlookup_127.29.194.20@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.29.194.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115480851507-30472-0/krb5kdc/rangerlookup_127.29.194.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.29.194.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.29.194.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.29.194.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115480851507-30472-0/krb5kdc/HTTP_127.29.194.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.29.194.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115480851507-30472-0/krb5kdc/HTTP_127.29.194.20@KRBTEST.COM.keytab.
I20260807 15:13:15.700632 30472 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115480851507-30472-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-task7fZSIL/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115480851507-30472-0/minicluster-data/postgres -l logfile start
2026-08-07 15:13:18.238 UTC [15415] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-07 15:13:18.238 UTC [15415] LOG: listening on IPv4 address "127.29.194.20", port 60889
2026-08-07 15:13:18.241 UTC [15415] LOG: listening on Unix socket "/tmp/.s.PGSQL.60889"
2026-08-07 15:13:18.270 UTC [15420] LOG: database system was shut down at 2026-08-07 15:13:16 UTC
2026-08-07 15:13:18.274 UTC [15415] LOG: database system is ready to accept connections
I20260807 15:13:19.330323 30472 mini_postgres.cc:96] Postgres bound to 60889
2026-08-07 15:13:19.338 UTC [15430] FATAL: database "slave" does not exist
127.29.194.20:60889 - accepting connections
I20260807 15:13:19.338743 30472 mini_ranger.cc:161] Starting Ranger...
I20260807 15:13:19.354938 30472 mini_ranger.cc:84] Created miniranger Postgres user
I20260807 15:13:19.842741 30472 mini_ranger.cc:87] Created ranger Postgres database
I20260807 15:13:19.842825 30472 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115480851507-30472-0/minicluster-data/ranger-admin
2026-08-07 15:13:20,137 [I] DB FLAVOR :POSTGRES
2026-08-07 15:13:20,137 [I] --------- Verifying Ranger DB connection ---------
2026-08-07 15:13:20,137 [I] Checking connection..
2026-08-07 15:13:20,137 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task7fZSIL/build/release/bin/postgresql.jar:/tmp/dist-test-task7fZSIL/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.194.20:60889/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-07 15:13:20,400 [I] Checking connection passed.
2026-08-07 15:13:20,400 [I] --------- Verifying version history table ---------
2026-08-07 15:13:20,400 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task7fZSIL/build/release/bin/postgresql.jar:/tmp/dist-test-task7fZSIL/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.194.20:60889/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-07 15:13:20,666 [I] Table x_db_version_h does not exist in database ranger
2026-08-07 15:13:20,667 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task7fZSIL/build/release/bin/postgresql.jar:/tmp/dist-test-task7fZSIL/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.194.20:60889/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-07 15:13:20,920 [I] Table x_db_version_h does not exist in database ranger
2026-08-07 15:13:20,921 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-07 15:13:20,921 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task7fZSIL/build/release/bin/postgresql.jar:/tmp/dist-test-task7fZSIL/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.194.20:60889/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-task7fZSIL/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-07 15:13:21.192 UTC [15525] WARNING: there is no transaction in progress
2026-08-07 15:13:21,199 [I] create_dbversion_catalog.sql file imported successfully
2026-08-07 15:13:21,200 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task7fZSIL/build/release/bin/postgresql.jar:/tmp/dist-test-task7fZSIL/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.194.20:60889/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-07 15:13:21,477 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-07 15:13:21,477 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-07 15:13:21,477 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task7fZSIL/build/release/bin/postgresql.jar:/tmp/dist-test-task7fZSIL/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.194.20:60889/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-07 15:13:21,752 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task7fZSIL/build/release/bin/postgresql.jar:/tmp/dist-test-task7fZSIL/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.194.20:60889/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-07 15:13:22,029 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task7fZSIL/build/release/bin/postgresql.jar:/tmp/dist-test-task7fZSIL/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.194.20:60889/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-42z9.c.gcp-upstream.internal','N') ;"
2026-08-07 15:13:22,280 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task7fZSIL/build/release/bin/postgresql.jar:/tmp/dist-test-task7fZSIL/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.194.20:60889/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-07 15:13:22,541 [I] Table x_portal_user does not exist in database ranger
2026-08-07 15:13:22,542 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task7fZSIL/build/release/bin/postgresql.jar:/tmp/dist-test-task7fZSIL/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.194.20:60889/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-07 15:13:22,803 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-07 15:13:22,803 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task7fZSIL/build/release/bin/postgresql.jar:/tmp/dist-test-task7fZSIL/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.194.20:60889/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-07 15:13:23,073 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task7fZSIL/build/release/bin/postgresql.jar:/tmp/dist-test-task7fZSIL/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.194.20:60889/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-07 15:13:23,330 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-07 15:13:23,331 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task7fZSIL/build/release/bin/postgresql.jar:/tmp/dist-test-task7fZSIL/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.194.20:60889/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-task7fZSIL/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-07 15:13:24.410 UTC [15692] WARNING: there is no transaction in progress
2026-08-07 15:13:24.433 UTC [15692] WARNING: there is no transaction in progress
2026-08-07 15:13:24.452 UTC [15692] WARNING: there is no transaction in progress
2026-08-07 15:13:24.469 UTC [15692] WARNING: there is no transaction in progress
2026-08-07 15:13:24.491 UTC [15692] WARNING: there is no transaction in progress
2026-08-07 15:13:24.510 UTC [15692] WARNING: there is no transaction in progress
2026-08-07 15:13:24.569 UTC [15692] WARNING: there is no transaction in progress
2026-08-07 15:13:24.576 UTC [15692] WARNING: there is no transaction in progress
2026-08-07 15:13:24.592 UTC [15692] WARNING: there is no transaction in progress
2026-08-07 15:13:24.604 UTC [15692] WARNING: there is no transaction in progress
2026-08-07 15:13:24.616 UTC [15692] WARNING: there is no transaction in progress
2026-08-07 15:13:24.631 UTC [15692] WARNING: there is no transaction in progress
2026-08-07 15:13:24.643 UTC [15692] WARNING: there is no transaction in progress
2026-08-07 15:13:24.651 UTC [15692] WARNING: there is no transaction in progress
2026-08-07 15:13:24.659 UTC [15692] WARNING: there is no transaction in progress
2026-08-07 15:13:24.667 UTC [15692] WARNING: there is no transaction in progress
2026-08-07 15:13:25,420 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-07 15:13:25,420 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task7fZSIL/build/release/bin/postgresql.jar:/tmp/dist-test-task7fZSIL/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.194.20:60889/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-07 15:13:25,666 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-07 15:13:25,667 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task7fZSIL/build/release/bin/postgresql.jar:/tmp/dist-test-task7fZSIL/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.194.20:60889/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-07 15:13:25,949 [I] Table x_portal_user already exists in database 'ranger'
2026-08-07 15:13:25,949 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task7fZSIL/build/release/bin/postgresql.jar:/tmp/dist-test-task7fZSIL/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.194.20:60889/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-07 15:13:26,226 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-07 15:13:26,226 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task7fZSIL/build/release/bin/postgresql.jar:/tmp/dist-test-task7fZSIL/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.194.20:60889/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-07 15:13:26,477 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task7fZSIL/build/release/bin/postgresql.jar:/tmp/dist-test-task7fZSIL/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.194.20:60889/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-07 15:13:26,756 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task7fZSIL/build/release/bin/postgresql.jar:/tmp/dist-test-task7fZSIL/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.194.20:60889/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-42z9.c.gcp-upstream.internal';"
2026-08-07 15:13:27,028 [I] CORE_DB_SCHEMA import status has been updated
2026-08-07 15:13:27,029 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task7fZSIL/build/release/bin/postgresql.jar:/tmp/dist-test-task7fZSIL/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.194.20:60889/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-07 15:13:27,297 [I] DB_PATCHES have already been applied
I20260807 15:13:27.303304 30472 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115480851507-30472-0/minicluster-data/ranger-admin:/tmp/dist-test-task7fZSIL/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-task7fZSIL/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-task7fZSIL/build/release/bin/postgresql.jar:/tmp/dist-test-task7fZSIL/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260807 15:13:27.303385 30472 mini_ranger.cc:193] Using host: 127.29.194.20
I20260807 15:13:27.307422 30472 mini_ranger.cc:227] Ranger admin URL: http://127.29.194.20:40179
Aug 07, 2026 3:13:27 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 07, 2026 3:13:27 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 07, 2026 3:13:27 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115480851507-30472-0/minicluster-data/ranger-admin/ews/webapp
Aug 07, 2026 3:13:27 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115480851507-30472-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 07, 2026 3:13:27 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115480851507-30472-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 07, 2026 3:13:28 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115480851507-30472-0/minicluster-data/ranger-admin/ews/webapp].
Aug 07, 2026 3:13:28 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260807 15:13:28.522423 30472 mini_ranger.cc:160] Time spent starting Ranger: real 9.184s user 0.001s sys 0.010s
Aug 07, 2026 3:13:49 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 07, 2026 3:13:49 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.UserREST
Aug 07, 2026 3:13:49 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 07, 2026 3:13:49 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 07, 2026 3:13:49 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 07, 2026 3:13:49 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 07, 2026 3:13:49 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 07, 2026 3:13:49 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 07, 2026 3:13:49 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 07, 2026 3:13:49 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 07, 2026 3:13:49 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 07, 2026 3:13:49 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 07, 2026 3:13:49 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 07, 2026 3:13:49 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 07, 2026 3:13:49 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 07, 2026 3:13:49 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 07, 2026 3:13:49 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 07, 2026 3:13:50 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260807 15:13:50.879739 30472 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.29.194.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.29.194.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.29.194.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115480851507-30472-0/krb5kdc/rangerkms_127.29.194.20@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.29.194.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115480851507-30472-0/krb5kdc/rangerkms_127.29.194.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.29.194.20@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.29.194.20@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.29.194.20@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115480851507-30472-0/krb5kdc/HTTP_127.29.194.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.29.194.20@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115480851507-30472-0/krb5kdc/HTTP_127.29.194.20@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 07 15:13:50 dist-test-slave-42z9 krb5kdc[15367](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115630, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260807 15:13:50.984990 30472 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260807 15:13:50.996727 30472 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260807 15:13:51.977666 30472 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260807 15:13:51.977749 30472 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115480851507-30472-0/minicluster-data/ranger-kms
I20260807 15:13:51.977761 30472 mini_ranger_kms.cc:227] Using postgres at 127.29.194.20:60889
2026-08-07 15:13:52,140 [I] DB FLAVOR :POSTGRES
2026-08-07 15:13:52,140 [I] --------- Verifying Ranger DB connection ---------
2026-08-07 15:13:52,140 [I] Checking connection
2026-08-07 15:13:52,140 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task7fZSIL/build/release/bin/postgresql.jar:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115480851507-30472-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.194.20:60889/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-07 15:13:52,408 [I] connection success
2026-08-07 15:13:52,408 [I] --------- Verifying Ranger DB tables ---------
2026-08-07 15:13:52,408 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-07 15:13:52,408 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task7fZSIL/build/release/bin/postgresql.jar:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115480851507-30472-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.194.20:60889/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-07 15:13:52,668 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-07 15:13:52,668 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-07 15:13:52,668 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-07 15:13:52,668 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task7fZSIL/build/release/bin/postgresql.jar:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115480851507-30472-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.194.20:60889/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115480851507-30472-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-07 15:13:52,960 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260807 15:13:53.508844 30472 mini_ranger_kms.cc:310] Created kms service
I20260807 15:13:53.729187 30472 mini_ranger_kms.cc:326] Created kudu user
I20260807 15:13:53.782137 30472 mini_ranger_kms.cc:343] Created rangerkms user
Aug 07 15:13:53 dist-test-slave-42z9 krb5kdc[15367](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115630, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.29.194.20@KRBTEST.COM
I20260807 15:13:54.247869 30472 mini_ranger_kms.cc:384] Added ranger policy
I20260807 15:13:54.248103 30472 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115480851507-30472-0/minicluster-data/ranger-kms:/tmp/dist-test-task7fZSIL/build/release/bin/postgresql.jar:/tmp/dist-test-task7fZSIL/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-task7fZSIL/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260807 15:13:54.248126 30472 mini_ranger_kms.cc:240] Using host: 127.29.194.20
I20260807 15:13:54.252288 30472 mini_ranger_kms.cc:276] Ranger KMS PID: 15978
I20260807 15:13:54.252393 30472 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.29.194.20:35045
Aug 07, 2026 3:13:55 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 07, 2026 3:13:55 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 07, 2026 3:13:55 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115480851507-30472-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 07, 2026 3:13:55 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115480851507-30472-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 07, 2026 3:13:55 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115480851507-30472-0/minicluster-data/ranger-kms/ews/webapp].
Aug 07, 2026 3:13:55 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 07 15:13:56 dist-test-slave-42z9 krb5kdc[15367](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786115636, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.29.194.20@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:13:56.707396 30472 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.722s user 0.005s sys 0.143s
I20260807 15:13:56.707513 30472 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260807 15:13:56.707599 30472 mini_ranger_kms.cc:401] 127.29.194.20:35045/kms/v1/keys
Aug 07 15:13:56 dist-test-slave-42z9 krb5kdc[15367](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115636, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.29.194.20@KRBTEST.COM for HTTP/127.29.194.20@KRBTEST.COM
Aug 07 15:13:57 dist-test-slave-42z9 krb5kdc[15367](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115636, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.29.194.20@KRBTEST.COM for HTTP/127.29.194.20@KRBTEST.COM
Aug 07 15:13:59 dist-test-slave-42z9 krb5kdc[15367](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115639, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.29.194.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.194.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.194.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.62.keytab.
Entry for principal kudu/127.29.194.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.62.keytab.
I20260807 15:13:59.792769 30472 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7fZSIL/build/release/bin/kudu
/tmp/dist-test-task7fZSIL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115480851507-30472-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115480851507-30472-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115480851507-30472-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115480851507-30472-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.194.62:41901
--webserver_interface=127.29.194.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.62.keytab
--principal=kudu/127.29.194.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.194.20:40769
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.194.62:41901
--ranger_config_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115480851507-30472-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.29.194.20:35045/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115480851507-30472-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115480851507-30472-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115480851507-30472-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:13:59.894510 16060 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:13:59.894755 16060 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:13:59.894780 16060 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:13:59.896051 16060 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:13:59.896090 16060 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:13:59.896103 16060 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:13:59.896113 16060 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:13:59.896124 16060 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:13:59.897781 16060 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.194.20:40769
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115480851507-30472-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115480851507-30472-0/minicluster-data/master-0/wal
--ranger_kms_url=127.29.194.20:35045/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.29.194.62:41901
--ranger_config_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115480851507-30472-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.62.keytab
--principal=kudu/127.29.194.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.194.62:41901
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115480851507-30472-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.194.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16060
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115480851507-30472-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:13:59.897997 16060 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:13:59.898187 16060 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:13:59.904261 16070 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:13:59.906332 16069 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:13:59.906415 16072 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:13:59.906589 16060 server_base.cc:1034] running on GCE node
I20260807 15:13:59.906857 16060 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:13:59.907167 16060 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:13:59.908344 16060 hybrid_clock.cc:648] HybridClock initialized: now 1786115639908300 us; error 69 us; skew 500 ppm
Aug 07 15:13:59 dist-test-slave-42z9 krb5kdc[15367](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115639, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.194.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:13:59.914150 16060 init.cc:377] Logged in from keytab as kudu/127.29.194.62@KRBTEST.COM (short username kudu)
I20260807 15:13:59.914642 16060 webserver.cc:460] Webserver started at http://127.29.194.62:32837/ using document root <none> and password file <none>
I20260807 15:13:59.914850 16060 fs_manager.cc:359] Metadata directory not provided
I20260807 15:13:59.914894 16060 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:13:59.915035 16060 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 15:13:59 dist-test-slave-42z9 krb5kdc[15367](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115639, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.194.62@KRBTEST.COM for HTTP/127.29.194.20@KRBTEST.COM
I20260807 15:13:59.965968 16060 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115480851507-30472-0/minicluster-data/master-0/data/instance:
uuid: "9739f9d5895d48248a63bcfa20d35884"
format_stamp: "Formatted at 2026-08-07 15:13:59 on dist-test-slave-42z9"
server_key: "951cbe9a11f4d13362667a4a8721439c"
server_key_iv: "5324b0a2de7f135989bb47614121e3b7"
server_key_version: "kuduclusterkey@0"
I20260807 15:13:59.966385 16060 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115480851507-30472-0/minicluster-data/master-0/wal/instance:
uuid: "9739f9d5895d48248a63bcfa20d35884"
format_stamp: "Formatted at 2026-08-07 15:13:59 on dist-test-slave-42z9"
server_key: "951cbe9a11f4d13362667a4a8721439c"
server_key_iv: "5324b0a2de7f135989bb47614121e3b7"
server_key_version: "kuduclusterkey@0"
I20260807 15:13:59.967875 16060 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 15:13:59.984839 16082 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:13:59.985358 16060 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:13:59.985577 16060 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115480851507-30472-0/minicluster-data/master-0/data,/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115480851507-30472-0/minicluster-data/master-0/wal
uuid: "9739f9d5895d48248a63bcfa20d35884"
format_stamp: "Formatted at 2026-08-07 15:13:59 on dist-test-slave-42z9"
server_key: "951cbe9a11f4d13362667a4a8721439c"
server_key_iv: "5324b0a2de7f135989bb47614121e3b7"
server_key_version: "kuduclusterkey@0"
I20260807 15:13:59.985720 16060 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115480851507-30472-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115480851507-30472-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115480851507-30472-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:13:59.995492 16060 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:13:59.997339 16060 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:13:59.997516 16060 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:14:00.002755 16060 rpc_server.cc:307] RPC server started. Bound to: 127.29.194.62:41901
I20260807 15:14:00.002805 16162 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.194.62:41901 every 8 connection(s)
I20260807 15:14:00.003188 16060 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115480851507-30472-0/minicluster-data/master-0/data/info.pb
I20260807 15:14:00.003925 16165 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:14:00.006275 16165 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9739f9d5895d48248a63bcfa20d35884: Bootstrap starting.
I20260807 15:14:00.006884 16165 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9739f9d5895d48248a63bcfa20d35884: Neither blocks nor log segments found. Creating new log.
I20260807 15:14:00.007160 16165 log.cc:824] T 00000000000000000000000000000000 P 9739f9d5895d48248a63bcfa20d35884: Log is configured to *not* fsync() on all Append() calls
I20260807 15:14:00.007869 16165 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9739f9d5895d48248a63bcfa20d35884: No bootstrap required, opened a new log
I20260807 15:14:00.009974 16165 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9739f9d5895d48248a63bcfa20d35884 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9739f9d5895d48248a63bcfa20d35884" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 41901 } }
I20260807 15:14:00.010134 16165 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9739f9d5895d48248a63bcfa20d35884 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:14:00.010159 16165 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9739f9d5895d48248a63bcfa20d35884 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9739f9d5895d48248a63bcfa20d35884, State: Initialized, Role: FOLLOWER
I20260807 15:14:00.010289 16165 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9739f9d5895d48248a63bcfa20d35884 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9739f9d5895d48248a63bcfa20d35884" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 41901 } }
I20260807 15:14:00.010361 16165 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9739f9d5895d48248a63bcfa20d35884 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:14:00.010385 16165 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9739f9d5895d48248a63bcfa20d35884 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:14:00.010412 16165 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9739f9d5895d48248a63bcfa20d35884 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:14:00.010983 16165 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9739f9d5895d48248a63bcfa20d35884 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9739f9d5895d48248a63bcfa20d35884" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 41901 } }
I20260807 15:14:00.011077 16165 leader_election.cc:302] T 00000000000000000000000000000000 P 9739f9d5895d48248a63bcfa20d35884 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9739f9d5895d48248a63bcfa20d35884; no voters:
I20260807 15:14:00.011243 16165 leader_election.cc:288] T 00000000000000000000000000000000 P 9739f9d5895d48248a63bcfa20d35884 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 15:14:00.011289 16169 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9739f9d5895d48248a63bcfa20d35884 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:14:00.011549 30472 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7fZSIL/build/release/bin/kudu as pid 16060
I20260807 15:14:00.011638 30472 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115480851507-30472-0/minicluster-data/master-0/wal/instance
I20260807 15:14:00.011802 30472 external_mini_cluster.cc:1428] Setting key bf3694b03bdefb19484c5060ad0b69b6
I20260807 15:14:00.011497 16165 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9739f9d5895d48248a63bcfa20d35884 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:14:00.012099 16169 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9739f9d5895d48248a63bcfa20d35884 [term 1 LEADER]: Becoming Leader. State: Replica: 9739f9d5895d48248a63bcfa20d35884, State: Running, Role: LEADER
I20260807 15:14:00.012221 16169 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9739f9d5895d48248a63bcfa20d35884 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9739f9d5895d48248a63bcfa20d35884" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 41901 } }
I20260807 15:14:00.012681 16172 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9739f9d5895d48248a63bcfa20d35884 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9739f9d5895d48248a63bcfa20d35884. Latest consensus state: current_term: 1 leader_uuid: "9739f9d5895d48248a63bcfa20d35884" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9739f9d5895d48248a63bcfa20d35884" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 41901 } } }
I20260807 15:14:00.012923 16172 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9739f9d5895d48248a63bcfa20d35884 [sys.catalog]: This master's current role is: LEADER
I20260807 15:14:00.013378 16170 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9739f9d5895d48248a63bcfa20d35884 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9739f9d5895d48248a63bcfa20d35884" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9739f9d5895d48248a63bcfa20d35884" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 41901 } } }
I20260807 15:14:00.013427 16185 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:14:00.013724 16165 ranger_client.cc:318] Using new properties file: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115480851507-30472-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260807 15:14:00.014185 16170 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9739f9d5895d48248a63bcfa20d35884 [sys.catalog]: This master's current role is: LEADER
I20260807 15:14:00.016793 16185 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 07 15:14:00 dist-test-slave-42z9 krb5kdc[15367](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115639, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.194.62@KRBTEST.COM
I20260807 15:14:00.022711 16185 catalog_manager.cc:1347] Generated new cluster ID: 2f1f6009f2cb42f78c1abd220c76beb4
I20260807 15:14:00.022843 16185 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:14:00.026969 16185 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:14:00.027179 16179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:00.012501 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42936 (local address 127.29.194.62:41901)
0807 15:14:00.012655 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0807 15:14:00.012665 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 15:14:00.012695 (+ 30us) server_negotiation.cc:407] Connection header received
0807 15:14:00.012817 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:14:00.012824 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:14:00.012895 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:14:00.013938 (+ 1043us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:14:00.014044 (+ 106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:00.018627 (+ 4583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:00.019624 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:00.020059 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:00.020162 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:00.022832 (+ 2670us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:14:00.022836 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:14:00.022837 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:14:00.022850 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:14:00.025231 (+ 2381us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:14:00.025709 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:14:00.025713 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:14:00.025713 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:14:00.025826 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:14:00.026015 (+ 189us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:14:00.026015 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:14:00.026016 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:14:00.026320 (+ 304us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:14:00.026455 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0807 15:14:00.026719 (+ 264us) server_negotiation.cc:299] Negotiation successful
0807 15:14:00.026858 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":98,"thread_start_us":74,"threads_started":1}
I20260807 15:14:00.027733 16185 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:14:00.035318 16185 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9739f9d5895d48248a63bcfa20d35884: Generated new TSK 0
I20260807 15:14:00.035648 16185 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.29.194.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.194.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.194.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.1.keytab.
Entry for principal kudu/127.29.194.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.1.keytab.
I20260807 15:14:00.804580 30472 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7fZSIL/build/release/bin/kudu
/tmp/dist-test-task7fZSIL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115480851507-30472-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115480851507-30472-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115480851507-30472-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115480851507-30472-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.29.194.20:35045/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.194.1:0
--local_ip_for_outbound_sockets=127.29.194.1
--webserver_interface=127.29.194.1
--webserver_port=0
--tserver_master_addrs=127.29.194.62:41901
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.1.keytab
--principal=kudu/127.29.194.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.194.20:40769
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115480851507-30472-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115480851507-30472-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115480851507-30472-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:14:00.929509 16228 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:14:00.929818 16228 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:14:00.929855 16228 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:14:00.931705 16228 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:14:00.931785 16228 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:14:00.931830 16228 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.194.1
I20260807 15:14:00.934185 16228 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.194.20:40769
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115480851507-30472-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115480851507-30472-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.29.194.20:35045/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.1.keytab
--principal=kudu/127.29.194.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.194.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115480851507-30472-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.194.1
--webserver_port=0
--tserver_master_addrs=127.29.194.62:41901
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16228
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.194.1
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115480851507-30472-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:14:00.934490 16228 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:14:00.934837 16228 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 15:14:00.940573 16228 server_base.cc:1034] running on GCE node
W20260807 15:14:00.940668 16241 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:14:00.940521 16237 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:14:00.940521 16236 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:14:00.941058 16228 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:14:00.941323 16228 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:14:00.942560 16228 hybrid_clock.cc:648] HybridClock initialized: now 1786115640942535 us; error 47 us; skew 500 ppm
Aug 07 15:14:00 dist-test-slave-42z9 krb5kdc[15367](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115640, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.194.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:14:00.946249 16228 init.cc:377] Logged in from keytab as kudu/127.29.194.1@KRBTEST.COM (short username kudu)
I20260807 15:14:00.946911 16228 webserver.cc:460] Webserver started at http://127.29.194.1:35015/ using document root <none> and password file <none>
I20260807 15:14:00.947232 16228 fs_manager.cc:359] Metadata directory not provided
I20260807 15:14:00.947300 16228 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:14:00.947455 16228 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 15:14:00 dist-test-slave-42z9 krb5kdc[15367](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115640, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.194.1@KRBTEST.COM for HTTP/127.29.194.20@KRBTEST.COM
I20260807 15:14:00.965917 16228 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115480851507-30472-0/minicluster-data/ts-0/data/instance:
uuid: "de619175db0a4b49946a0f3a5cb7f5d7"
format_stamp: "Formatted at 2026-08-07 15:14:00 on dist-test-slave-42z9"
server_key: "4db00b9281c949c8e4fb1939cda38fcc"
server_key_iv: "813b0c3cea0811cc0377261aca0e07e1"
server_key_version: "kuduclusterkey@0"
I20260807 15:14:00.966595 16228 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115480851507-30472-0/minicluster-data/ts-0/wal/instance:
uuid: "de619175db0a4b49946a0f3a5cb7f5d7"
format_stamp: "Formatted at 2026-08-07 15:14:00 on dist-test-slave-42z9"
server_key: "4db00b9281c949c8e4fb1939cda38fcc"
server_key_iv: "813b0c3cea0811cc0377261aca0e07e1"
server_key_version: "kuduclusterkey@0"
I20260807 15:14:00.967975 16228 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:14:00.981608 16253 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:14:00.981850 16228 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260807 15:14:00.981963 16228 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115480851507-30472-0/minicluster-data/ts-0/data,/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115480851507-30472-0/minicluster-data/ts-0/wal
uuid: "de619175db0a4b49946a0f3a5cb7f5d7"
format_stamp: "Formatted at 2026-08-07 15:14:00 on dist-test-slave-42z9"
server_key: "4db00b9281c949c8e4fb1939cda38fcc"
server_key_iv: "813b0c3cea0811cc0377261aca0e07e1"
server_key_version: "kuduclusterkey@0"
I20260807 15:14:00.982057 16228 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115480851507-30472-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115480851507-30472-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115480851507-30472-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:14:00.994611 16228 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:14:00.997260 16228 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:14:00.997537 16228 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:14:00.997790 16228 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:14:00.998263 16228 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:14:00.998359 16228 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:14:00.998467 16228 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:14:00.998539 16228 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:14:01.005322 16228 rpc_server.cc:307] RPC server started. Bound to: 127.29.194.1:40479
I20260807 15:14:01.005918 16228 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115480851507-30472-0/minicluster-data/ts-0/data/info.pb
I20260807 15:14:01.008734 16436 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.194.1:40479 every 8 connection(s)
I20260807 15:14:01.013720 30472 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7fZSIL/build/release/bin/kudu as pid 16228
I20260807 15:14:01.013863 30472 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115480851507-30472-0/minicluster-data/ts-0/wal/instance
I20260807 15:14:01.014089 30472 external_mini_cluster.cc:1428] Setting key 679a21b8abe363e2ced13313e789a5e6
Aug 07 15:14:01 dist-test-slave-42z9 krb5kdc[15367](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115640, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.194.1@KRBTEST.COM for kudu/127.29.194.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.29.194.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.194.2@KRBTEST.COM" created.
I20260807 15:14:01.034052 16443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:01.009695 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.194.1:45099 (local address 127.29.194.62:41901)
0807 15:14:01.011657 (+ 1962us) server_negotiation.cc:206] Beginning negotiation
0807 15:14:01.011659 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:14:01.011670 (+ 11us) server_negotiation.cc:407] Connection header received
0807 15:14:01.011699 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:14:01.011700 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:14:01.011714 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:14:01.011805 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:14:01.012745 (+ 940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:01.013661 (+ 916us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:01.015255 (+ 1594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:01.015563 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:01.015624 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:01.018515 (+ 2891us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:14:01.018517 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:14:01.018517 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:14:01.018526 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:14:01.026344 (+ 7818us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:14:01.032585 (+ 6241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:14:01.032587 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:14:01.032588 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:14:01.032670 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:14:01.033024 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:14:01.033026 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:14:01.033026 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:14:01.033211 (+ 185us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:14:01.033299 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0807 15:14:01.033787 (+ 488us) server_negotiation.cc:299] Negotiation successful
0807 15:14:01.033915 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1927,"thread_start_us":87,"threads_started":1}
I20260807 15:14:01.034180 16442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:01.009393 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:41901 (local address 127.29.194.1:45099)
0807 15:14:01.009588 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0807 15:14:01.009631 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0807 15:14:01.010413 (+ 782us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:14:01.012004 (+ 1591us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:14:01.012010 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:14:01.012290 (+ 280us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:14:01.012602 (+ 312us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:14:01.012610 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:14:01.013789 (+ 1179us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:14:01.013790 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:14:01.015145 (+ 1355us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:14:01.015149 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:14:01.015713 (+ 564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:14:01.015714 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:14:01.015875 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:01.016528 (+ 653us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:14:01.016540 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:14:01.018380 (+ 1840us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:14:01.032035 (+ 13655us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:14:01.032050 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:14:01.032054 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:14:01.032426 (+ 372us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:14:01.032743 (+ 317us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:14:01.032744 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:14:01.032744 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:14:01.032893 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:14:01.033303 (+ 410us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:14:01.033311 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:14:01.033591 (+ 280us) client_negotiation.cc:769] Sending connection context
0807 15:14:01.033718 (+ 127us) client_negotiation.cc:240] Negotiation successful
0807 15:14:01.033916 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":119,"threads_started":1}
I20260807 15:14:01.035128 16438 heartbeater.cc:347] Connected to a master server at 127.29.194.62:41901
I20260807 15:14:01.035247 16438 heartbeater.cc:464] Registering TS with master...
I20260807 15:14:01.035486 16438 heartbeater.cc:511] Master 127.29.194.62:41901 requested a full tablet report, sending...
I20260807 15:14:01.036087 16108 ts_manager.cc:194] Registered new tserver with Master: de619175db0a4b49946a0f3a5cb7f5d7 (127.29.194.1:40479)
I20260807 15:14:01.036998 16108 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.194.1@KRBTEST.COM'} at 127.29.194.1:45099
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.194.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.2.keytab.
Entry for principal kudu/127.29.194.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.2.keytab.
I20260807 15:14:01.048452 30472 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7fZSIL/build/release/bin/kudu
/tmp/dist-test-task7fZSIL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115480851507-30472-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115480851507-30472-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115480851507-30472-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115480851507-30472-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.29.194.20:35045/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.194.2:0
--local_ip_for_outbound_sockets=127.29.194.2
--webserver_interface=127.29.194.2
--webserver_port=0
--tserver_master_addrs=127.29.194.62:41901
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.2.keytab
--principal=kudu/127.29.194.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.194.20:40769
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115480851507-30472-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115480851507-30472-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115480851507-30472-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:14:01.169399 16447 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:14:01.169725 16447 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:14:01.169761 16447 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:14:01.171221 16447 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:14:01.171273 16447 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:14:01.171314 16447 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.194.2
I20260807 15:14:01.173139 16447 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.194.20:40769
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115480851507-30472-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115480851507-30472-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.29.194.20:35045/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.2.keytab
--principal=kudu/127.29.194.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.194.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115480851507-30472-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.194.2
--webserver_port=0
--tserver_master_addrs=127.29.194.62:41901
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16447
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.194.2
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115480851507-30472-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:14:01.173408 16447 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:14:01.173709 16447 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:14:01.178890 16455 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:14:01.178894 16457 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:14:01.179255 16454 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:14:01.180341 16447 server_base.cc:1034] running on GCE node
I20260807 15:14:01.180570 16447 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:14:01.180857 16447 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:14:01.182037 16447 hybrid_clock.cc:648] HybridClock initialized: now 1786115641182019 us; error 51 us; skew 500 ppm
Aug 07 15:14:01 dist-test-slave-42z9 krb5kdc[15367](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115641, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.194.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:14:01.184648 16447 init.cc:377] Logged in from keytab as kudu/127.29.194.2@KRBTEST.COM (short username kudu)
I20260807 15:14:01.185158 16447 webserver.cc:460] Webserver started at http://127.29.194.2:36699/ using document root <none> and password file <none>
I20260807 15:14:01.185379 16447 fs_manager.cc:359] Metadata directory not provided
I20260807 15:14:01.185431 16447 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:14:01.185606 16447 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 15:14:01 dist-test-slave-42z9 krb5kdc[15367](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115641, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.194.2@KRBTEST.COM for HTTP/127.29.194.20@KRBTEST.COM
I20260807 15:14:01.201242 16447 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115480851507-30472-0/minicluster-data/ts-1/data/instance:
uuid: "4709a7d453c74a9f88670579db66b8b8"
format_stamp: "Formatted at 2026-08-07 15:14:01 on dist-test-slave-42z9"
server_key: "da2050cb30663bc65f595ff6617e67b8"
server_key_iv: "af7053b2ca414edbd1156f670ee553f0"
server_key_version: "kuduclusterkey@0"
I20260807 15:14:01.201857 16447 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115480851507-30472-0/minicluster-data/ts-1/wal/instance:
uuid: "4709a7d453c74a9f88670579db66b8b8"
format_stamp: "Formatted at 2026-08-07 15:14:01 on dist-test-slave-42z9"
server_key: "da2050cb30663bc65f595ff6617e67b8"
server_key_iv: "af7053b2ca414edbd1156f670ee553f0"
server_key_version: "kuduclusterkey@0"
I20260807 15:14:01.208585 16447 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.000s
Aug 07 15:14:01 dist-test-slave-42z9 krb5kdc[15367](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786115641, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.194.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:14:01.238148 16474 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:14:01.238425 16447 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.002s
I20260807 15:14:01.238526 16447 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115480851507-30472-0/minicluster-data/ts-1/data,/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115480851507-30472-0/minicluster-data/ts-1/wal
uuid: "4709a7d453c74a9f88670579db66b8b8"
format_stamp: "Formatted at 2026-08-07 15:14:01 on dist-test-slave-42z9"
server_key: "da2050cb30663bc65f595ff6617e67b8"
server_key_iv: "af7053b2ca414edbd1156f670ee553f0"
server_key_version: "kuduclusterkey@0"
I20260807 15:14:01.238618 16447 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115480851507-30472-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115480851507-30472-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115480851507-30472-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:14:01.292363 16447 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:14:01.294948 16447 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:14:01.295188 16447 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:14:01.295470 16447 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:14:01.295953 16447 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:14:01.296051 16447 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:14:01.296126 16447 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:14:01.296185 16447 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:14:01.303603 16447 rpc_server.cc:307] RPC server started. Bound to: 127.29.194.2:46047
I20260807 15:14:01.303802 16646 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.194.2:46047 every 8 connection(s)
I20260807 15:14:01.304152 16447 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115480851507-30472-0/minicluster-data/ts-1/data/info.pb
Aug 07 15:14:01 dist-test-slave-42z9 krb5kdc[15367](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115641, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.194.2@KRBTEST.COM for kudu/127.29.194.62@KRBTEST.COM
I20260807 15:14:01.313917 30472 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7fZSIL/build/release/bin/kudu as pid 16447
I20260807 15:14:01.314034 30472 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115480851507-30472-0/minicluster-data/ts-1/wal/instance
I20260807 15:14:01.314409 30472 external_mini_cluster.cc:1428] Setting key f00a7ae11a4c11ec757375dc4b544d92
I20260807 15:14:01.316619 16443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:01.305014 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.194.2:58967 (local address 127.29.194.62:41901)
0807 15:14:01.305080 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0807 15:14:01.305082 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:14:01.305247 (+ 165us) server_negotiation.cc:407] Connection header received
0807 15:14:01.305877 (+ 630us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:14:01.305879 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:14:01.305897 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:14:01.305951 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:14:01.307152 (+ 1201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:01.307741 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:01.308908 (+ 1167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:01.309134 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:01.309194 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:01.311891 (+ 2697us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:14:01.311893 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:14:01.311893 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:14:01.311903 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:14:01.313350 (+ 1447us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:14:01.315273 (+ 1923us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:14:01.315275 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:14:01.315275 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:14:01.315359 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:14:01.315715 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:14:01.315716 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:14:01.315716 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:14:01.315896 (+ 180us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:14:01.315951 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0807 15:14:01.316363 (+ 412us) server_negotiation.cc:299] Negotiation successful
0807 15:14:01.316511 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260807 15:14:01.317528 16650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:01.304937 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:41901 (local address 127.29.194.2:58967)
0807 15:14:01.305115 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0807 15:14:01.305159 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0807 15:14:01.305760 (+ 601us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:14:01.306035 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:14:01.306040 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:14:01.306252 (+ 212us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:14:01.306962 (+ 710us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:14:01.306976 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:14:01.307858 (+ 882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:14:01.307859 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:14:01.308802 (+ 943us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:14:01.308806 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:14:01.309232 (+ 426us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:14:01.309234 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:14:01.309395 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:01.310071 (+ 676us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:14:01.310081 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:14:01.311786 (+ 1705us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:14:01.313431 (+ 1645us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:14:01.313475 (+ 44us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:14:01.313478 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:14:01.315172 (+ 1694us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:14:01.315438 (+ 266us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:14:01.315439 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:14:01.315439 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:14:01.315621 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:14:01.315988 (+ 367us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:14:01.315995 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:14:01.316194 (+ 199us) client_negotiation.cc:769] Sending connection context
0807 15:14:01.316733 (+ 539us) client_negotiation.cc:240] Negotiation successful
0807 15:14:01.317186 (+ 453us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":94,"threads_started":1}
I20260807 15:14:01.318404 16648 heartbeater.cc:347] Connected to a master server at 127.29.194.62:41901
I20260807 15:14:01.318536 16648 heartbeater.cc:464] Registering TS with master...
I20260807 15:14:01.318782 16648 heartbeater.cc:511] Master 127.29.194.62:41901 requested a full tablet report, sending...
I20260807 15:14:01.319213 16108 ts_manager.cc:194] Registered new tserver with Master: 4709a7d453c74a9f88670579db66b8b8 (127.29.194.2:46047)
I20260807 15:14:01.319734 16108 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.194.2@KRBTEST.COM'} at 127.29.194.2:58967
WARNING: no policy specified for kudu/127.29.194.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.194.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.194.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.3.keytab.
Entry for principal kudu/127.29.194.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.3.keytab.
I20260807 15:14:01.349227 30472 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7fZSIL/build/release/bin/kudu
/tmp/dist-test-task7fZSIL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115480851507-30472-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115480851507-30472-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115480851507-30472-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115480851507-30472-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.29.194.20:35045/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.194.3:0
--local_ip_for_outbound_sockets=127.29.194.3
--webserver_interface=127.29.194.3
--webserver_port=0
--tserver_master_addrs=127.29.194.62:41901
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.3.keytab
--principal=kudu/127.29.194.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.194.20:40769
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115480851507-30472-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115480851507-30472-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115480851507-30472-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:14:01.477317 16655 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:14:01.477674 16655 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:14:01.477715 16655 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:14:01.479702 16655 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:14:01.479792 16655 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:14:01.479841 16655 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.194.3
I20260807 15:14:01.482439 16655 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.194.20:40769
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115480851507-30472-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115480851507-30472-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.29.194.20:35045/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.3.keytab
--principal=kudu/127.29.194.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.194.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115480851507-30472-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.194.3
--webserver_port=0
--tserver_master_addrs=127.29.194.62:41901
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16655
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.194.3
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115480851507-30472-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:14:01.482916 16655 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:14:01.483357 16655 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:14:01.490478 16668 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:14:01.490890 16669 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:14:01.491138 16675 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:14:01.491760 16655 server_base.cc:1034] running on GCE node
I20260807 15:14:01.492128 16655 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:14:01.492400 16655 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:14:01.493561 16655 hybrid_clock.cc:648] HybridClock initialized: now 1786115641493536 us; error 40 us; skew 500 ppm
Aug 07 15:14:01 dist-test-slave-42z9 krb5kdc[15367](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115641, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.194.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:14:01.497040 16655 init.cc:377] Logged in from keytab as kudu/127.29.194.3@KRBTEST.COM (short username kudu)
I20260807 15:14:01.497578 16655 webserver.cc:460] Webserver started at http://127.29.194.3:34705/ using document root <none> and password file <none>
I20260807 15:14:01.497813 16655 fs_manager.cc:359] Metadata directory not provided
I20260807 15:14:01.497864 16655 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:14:01.498000 16655 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 15:14:01 dist-test-slave-42z9 krb5kdc[15367](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115641, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.194.3@KRBTEST.COM for HTTP/127.29.194.20@KRBTEST.COM
I20260807 15:14:01.511346 16655 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115480851507-30472-0/minicluster-data/ts-2/data/instance:
uuid: "390340d1b4064577acddca506f6732d4"
format_stamp: "Formatted at 2026-08-07 15:14:01 on dist-test-slave-42z9"
server_key: "29f256276d753804cc09cc11bccd0400"
server_key_iv: "7e49721e977878c84091fcac3316b51b"
server_key_version: "kuduclusterkey@0"
I20260807 15:14:01.511819 16655 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115480851507-30472-0/minicluster-data/ts-2/wal/instance:
uuid: "390340d1b4064577acddca506f6732d4"
format_stamp: "Formatted at 2026-08-07 15:14:01 on dist-test-slave-42z9"
server_key: "29f256276d753804cc09cc11bccd0400"
server_key_iv: "7e49721e977878c84091fcac3316b51b"
server_key_version: "kuduclusterkey@0"
I20260807 15:14:01.513608 16655 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20260807 15:14:01.539105 16693 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:14:01.539312 16655 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:14:01.539427 16655 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115480851507-30472-0/minicluster-data/ts-2/data,/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115480851507-30472-0/minicluster-data/ts-2/wal
uuid: "390340d1b4064577acddca506f6732d4"
format_stamp: "Formatted at 2026-08-07 15:14:01 on dist-test-slave-42z9"
server_key: "29f256276d753804cc09cc11bccd0400"
server_key_iv: "7e49721e977878c84091fcac3316b51b"
server_key_version: "kuduclusterkey@0"
I20260807 15:14:01.539531 16655 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115480851507-30472-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115480851507-30472-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115480851507-30472-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:14:01.555002 16655 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:14:01.557586 16655 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:14:01.557840 16655 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:14:01.558158 16655 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:14:01.558619 16655 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:14:01.558709 16655 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:14:01.558784 16655 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:14:01.558844 16655 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:14:01.568010 16655 rpc_server.cc:307] RPC server started. Bound to: 127.29.194.3:45165
I20260807 15:14:01.568619 16655 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115480851507-30472-0/minicluster-data/ts-2/data/info.pb
I20260807 15:14:01.569530 16876 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.194.3:45165 every 8 connection(s)
Aug 07 15:14:01 dist-test-slave-42z9 krb5kdc[15367](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115641, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.194.3@KRBTEST.COM for kudu/127.29.194.62@KRBTEST.COM
I20260807 15:14:01.578027 30472 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7fZSIL/build/release/bin/kudu as pid 16655
I20260807 15:14:01.578143 30472 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115480851507-30472-0/minicluster-data/ts-2/wal/instance
I20260807 15:14:01.579618 30472 external_mini_cluster.cc:1428] Setting key 03d87c0d475f122ee623e63b96e72e2a
I20260807 15:14:01.582783 16443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:01.571692 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.194.3:56631 (local address 127.29.194.62:41901)
0807 15:14:01.571775 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0807 15:14:01.571777 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:14:01.571789 (+ 12us) server_negotiation.cc:407] Connection header received
0807 15:14:01.571819 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:14:01.571820 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:14:01.571832 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:14:01.571891 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:14:01.572805 (+ 914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:01.573647 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:01.575123 (+ 1476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:01.575401 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:01.575449 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:01.578324 (+ 2875us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:14:01.578326 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:14:01.578326 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:14:01.578335 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:14:01.580213 (+ 1878us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:14:01.581596 (+ 1383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:14:01.581597 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:14:01.581598 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:14:01.581666 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:14:01.581981 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:14:01.581982 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:14:01.581982 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:14:01.582134 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:14:01.582180 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0807 15:14:01.582586 (+ 406us) server_negotiation.cc:299] Negotiation successful
0807 15:14:01.582681 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260807 15:14:01.583583 16883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:01.569756 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:41901 (local address 127.29.194.3:56631)
0807 15:14:01.569931 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0807 15:14:01.569973 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0807 15:14:01.570602 (+ 629us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:14:01.572044 (+ 1442us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:14:01.572049 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:14:01.572363 (+ 314us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:14:01.572683 (+ 320us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:14:01.572690 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:14:01.573767 (+ 1077us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:14:01.573768 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:14:01.575016 (+ 1248us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:14:01.575020 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:14:01.575578 (+ 558us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:14:01.575579 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:14:01.575735 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:01.576341 (+ 606us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:14:01.576352 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:14:01.578161 (+ 1809us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:14:01.581126 (+ 2965us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:14:01.581135 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:14:01.581138 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:14:01.581493 (+ 355us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:14:01.581742 (+ 249us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:14:01.581743 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:14:01.581743 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:14:01.581902 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:14:01.582226 (+ 324us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:14:01.582232 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:14:01.582405 (+ 173us) client_negotiation.cc:769] Sending connection context
0807 15:14:01.583136 (+ 731us) client_negotiation.cc:240] Negotiation successful
0807 15:14:01.583324 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":87,"threads_started":1}
I20260807 15:14:01.584247 16877 heartbeater.cc:347] Connected to a master server at 127.29.194.62:41901
I20260807 15:14:01.584352 16877 heartbeater.cc:464] Registering TS with master...
I20260807 15:14:01.584564 16877 heartbeater.cc:511] Master 127.29.194.62:41901 requested a full tablet report, sending...
I20260807 15:14:01.584980 16108 ts_manager.cc:194] Registered new tserver with Master: 390340d1b4064577acddca506f6732d4 (127.29.194.3:45165)
I20260807 15:14:01.585482 16108 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.194.3@KRBTEST.COM'} at 127.29.194.3:56631
I20260807 15:14:01.591881 30472 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 15:14:01.592608 30472 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7fZSIL/build/release/bin/kudu with pid 16228
I20260807 15:14:01.635599 30472 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7fZSIL/build/release/bin/kudu with pid 16447
I20260807 15:14:01.683020 30472 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7fZSIL/build/release/bin/kudu with pid 16655
I20260807 15:14:01.721720 30472 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7fZSIL/build/release/bin/kudu with pid 16060
I20260807 15:14:01.742432 30472 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7fZSIL/build/release/bin/kudu
/tmp/dist-test-task7fZSIL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115480851507-30472-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115480851507-30472-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115480851507-30472-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115480851507-30472-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.194.62:41901
--webserver_interface=127.29.194.62
--webserver_port=32837
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.62.keytab
--principal=kudu/127.29.194.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.194.20:40769
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.194.62:41901
--ranger_config_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115480851507-30472-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.29.194.20:35045/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.29.194.20:35045/kms with env {KRB5CCNAME=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115480851507-30472-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115480851507-30472-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115480851507-30472-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:14:01.836505 16884 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:14:01.836726 16884 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:14:01.836748 16884 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:14:01.837885 16884 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:14:01.837921 16884 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:14:01.837934 16884 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:14:01.837945 16884 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:14:01.837956 16884 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:14:01.839562 16884 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.194.20:40769
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115480851507-30472-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115480851507-30472-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.29.194.20:35045/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.29.194.62:41901
--ranger_config_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115480851507-30472-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.62.keytab
--principal=kudu/127.29.194.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.194.62:41901
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115480851507-30472-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.194.62
--webserver_port=32837
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16884
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115480851507-30472-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:14:01.839766 16884 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:14:01.839942 16884 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 15:14:01.844839 16884 server_base.cc:1034] running on GCE node
W20260807 15:14:01.844843 16892 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:14:01.844806 16897 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:14:01.844806 16893 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:14:01.845300 16884 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:14:01.845636 16884 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:14:01.846779 16884 hybrid_clock.cc:648] HybridClock initialized: now 1786115641846766 us; error 28 us; skew 500 ppm
Aug 07 15:14:01 dist-test-slave-42z9 krb5kdc[15367](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115641, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.194.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:14:01.849676 16884 init.cc:377] Logged in from keytab as kudu/127.29.194.62@KRBTEST.COM (short username kudu)
I20260807 15:14:01.850355 16884 webserver.cc:460] Webserver started at http://127.29.194.62:32837/ using document root <none> and password file <none>
I20260807 15:14:01.850637 16884 fs_manager.cc:359] Metadata directory not provided
I20260807 15:14:01.850713 16884 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 07 15:14:01 dist-test-slave-42z9 krb5kdc[15367](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115641, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.194.62@KRBTEST.COM for HTTP/127.29.194.20@KRBTEST.COM
I20260807 15:14:01.923770 16884 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:14:01.924710 16912 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:14:01.924916 16884 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:14:01.925031 16884 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115480851507-30472-0/minicluster-data/master-0/data,/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115480851507-30472-0/minicluster-data/master-0/wal
uuid: "9739f9d5895d48248a63bcfa20d35884"
format_stamp: "Formatted at 2026-08-07 15:13:59 on dist-test-slave-42z9"
server_key: "951cbe9a11f4d13362667a4a8721439c"
server_key_iv: "5324b0a2de7f135989bb47614121e3b7"
server_key_version: "kuduclusterkey@0"
I20260807 15:14:01.925515 16884 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115480851507-30472-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115480851507-30472-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115480851507-30472-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:14:01.955578 16884 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:14:01.957551 16884 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:14:01.957691 16884 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:14:01.961709 16884 rpc_server.cc:307] RPC server started. Bound to: 127.29.194.62:41901
I20260807 15:14:01.961766 16985 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.194.62:41901 every 8 connection(s)
I20260807 15:14:01.962137 16884 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115480851507-30472-0/minicluster-data/master-0/data/info.pb
I20260807 15:14:01.963420 16986 sys_catalog.cc:263] Verifying existing consensus state
I20260807 15:14:01.964104 16986 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9739f9d5895d48248a63bcfa20d35884: Bootstrap starting.
I20260807 15:14:01.971395 30472 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7fZSIL/build/release/bin/kudu as pid 16884
I20260807 15:14:01.971725 30472 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7fZSIL/build/release/bin/kudu
/tmp/dist-test-task7fZSIL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115480851507-30472-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115480851507-30472-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115480851507-30472-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115480851507-30472-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.29.194.20:35045/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.194.1:40479
--local_ip_for_outbound_sockets=127.29.194.1
--tserver_master_addrs=127.29.194.62:41901
--webserver_port=35015
--webserver_interface=127.29.194.1
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.1.keytab
--principal=kudu/127.29.194.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.194.20:40769
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.29.194.20:35045/kms with env {KRB5CCNAME=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115480851507-30472-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115480851507-30472-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115480851507-30472-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 15:14:01.972218 16986 log.cc:824] T 00000000000000000000000000000000 P 9739f9d5895d48248a63bcfa20d35884: Log is configured to *not* fsync() on all Append() calls
I20260807 15:14:01.974627 16986 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9739f9d5895d48248a63bcfa20d35884: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 15:14:01.974932 16986 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9739f9d5895d48248a63bcfa20d35884: Bootstrap complete.
I20260807 15:14:01.978292 16986 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9739f9d5895d48248a63bcfa20d35884 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9739f9d5895d48248a63bcfa20d35884" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 41901 } }
I20260807 15:14:01.978662 16986 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9739f9d5895d48248a63bcfa20d35884 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9739f9d5895d48248a63bcfa20d35884, State: Initialized, Role: FOLLOWER
I20260807 15:14:01.978858 16986 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9739f9d5895d48248a63bcfa20d35884 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9739f9d5895d48248a63bcfa20d35884" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 41901 } }
I20260807 15:14:01.978940 16986 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9739f9d5895d48248a63bcfa20d35884 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:14:01.979004 16986 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9739f9d5895d48248a63bcfa20d35884 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:14:01.979058 16986 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9739f9d5895d48248a63bcfa20d35884 [term 1 FOLLOWER]: Advancing to term 2
I20260807 15:14:01.980643 16986 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9739f9d5895d48248a63bcfa20d35884 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9739f9d5895d48248a63bcfa20d35884" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 41901 } }
I20260807 15:14:01.980772 16986 leader_election.cc:302] T 00000000000000000000000000000000 P 9739f9d5895d48248a63bcfa20d35884 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9739f9d5895d48248a63bcfa20d35884; no voters:
I20260807 15:14:01.980966 16986 leader_election.cc:288] T 00000000000000000000000000000000 P 9739f9d5895d48248a63bcfa20d35884 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260807 15:14:01.981225 16990 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9739f9d5895d48248a63bcfa20d35884 [term 2 FOLLOWER]: Leader election won for term 2
I20260807 15:14:01.981284 16986 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9739f9d5895d48248a63bcfa20d35884 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:14:01.981515 16990 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9739f9d5895d48248a63bcfa20d35884 [term 2 LEADER]: Becoming Leader. State: Replica: 9739f9d5895d48248a63bcfa20d35884, State: Running, Role: LEADER
I20260807 15:14:01.981674 16990 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9739f9d5895d48248a63bcfa20d35884 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9739f9d5895d48248a63bcfa20d35884" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 41901 } }
I20260807 15:14:01.982076 16996 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9739f9d5895d48248a63bcfa20d35884 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9739f9d5895d48248a63bcfa20d35884. Latest consensus state: current_term: 2 leader_uuid: "9739f9d5895d48248a63bcfa20d35884" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9739f9d5895d48248a63bcfa20d35884" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 41901 } } }
I20260807 15:14:01.982162 16996 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9739f9d5895d48248a63bcfa20d35884 [sys.catalog]: This master's current role is: LEADER
I20260807 15:14:01.982566 16995 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9739f9d5895d48248a63bcfa20d35884 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "9739f9d5895d48248a63bcfa20d35884" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9739f9d5895d48248a63bcfa20d35884" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 41901 } } }
I20260807 15:14:01.982640 16995 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9739f9d5895d48248a63bcfa20d35884 [sys.catalog]: This master's current role is: LEADER
I20260807 15:14:01.982649 17008 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:14:01.983175 17008 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:14:01.983592 17008 catalog_manager.cc:1259] Loaded cluster ID: 2f1f6009f2cb42f78c1abd220c76beb4
I20260807 15:14:01.983646 16986 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115480851507-30472-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260807 15:14:01.983843 17008 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:14:01.985354 17008 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:14:01.985571 17008 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 9739f9d5895d48248a63bcfa20d35884: Loaded TSK: 0
I20260807 15:14:01.985782 17008 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260807 15:14:02.093164 16989 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:14:02.093602 16989 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:14:02.093698 16989 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:14:02.095564 16989 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:14:02.095705 16989 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:14:02.095817 16989 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.194.1
I20260807 15:14:02.098559 16989 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.194.20:40769
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115480851507-30472-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115480851507-30472-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.29.194.20:35045/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.1.keytab
--principal=kudu/127.29.194.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.194.1:40479
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115480851507-30472-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.194.1
--webserver_port=35015
--tserver_master_addrs=127.29.194.62:41901
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16989
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.194.1
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115480851507-30472-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:14:02.099115 16989 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:14:02.099516 16989 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:14:02.106490 17050 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:14:02.106703 16989 server_base.cc:1034] running on GCE node
W20260807 15:14:02.106926 17052 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:14:02.107139 17049 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:14:02.107410 16989 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:14:02.107709 16989 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:14:02.108870 16989 hybrid_clock.cc:648] HybridClock initialized: now 1786115642108844 us; error 43 us; skew 500 ppm
Aug 07 15:14:02 dist-test-slave-42z9 krb5kdc[15367](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115642, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.194.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:14:02.112282 16989 init.cc:377] Logged in from keytab as kudu/127.29.194.1@KRBTEST.COM (short username kudu)
I20260807 15:14:02.113091 16989 webserver.cc:460] Webserver started at http://127.29.194.1:35015/ using document root <none> and password file <none>
I20260807 15:14:02.113497 16989 fs_manager.cc:359] Metadata directory not provided
I20260807 15:14:02.113652 16989 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 07 15:14:02 dist-test-slave-42z9 krb5kdc[15367](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115642, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.194.1@KRBTEST.COM for HTTP/127.29.194.20@KRBTEST.COM
I20260807 15:14:02.133846 16989 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:14:02.138727 17063 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:14:02.138947 16989 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.000s sys 0.000s
I20260807 15:14:02.139041 16989 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115480851507-30472-0/minicluster-data/ts-0/data,/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115480851507-30472-0/minicluster-data/ts-0/wal
uuid: "de619175db0a4b49946a0f3a5cb7f5d7"
format_stamp: "Formatted at 2026-08-07 15:14:00 on dist-test-slave-42z9"
server_key: "4db00b9281c949c8e4fb1939cda38fcc"
server_key_iv: "813b0c3cea0811cc0377261aca0e07e1"
server_key_version: "kuduclusterkey@0"
I20260807 15:14:02.139556 16989 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115480851507-30472-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115480851507-30472-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115480851507-30472-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:14:02.153883 16989 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:14:02.156368 16989 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:14:02.156603 16989 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:14:02.156857 16989 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:14:02.157279 16989 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:14:02.157351 16989 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:14:02.157398 16989 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:14:02.157431 16989 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:14:02.165330 16989 rpc_server.cc:307] RPC server started. Bound to: 127.29.194.1:40479
I20260807 15:14:02.166023 16989 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115480851507-30472-0/minicluster-data/ts-0/data/info.pb
I20260807 15:14:02.169027 17232 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.194.1:40479 every 8 connection(s)
I20260807 15:14:02.171949 30472 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7fZSIL/build/release/bin/kudu as pid 16989
I20260807 15:14:02.172261 30472 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7fZSIL/build/release/bin/kudu
/tmp/dist-test-task7fZSIL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115480851507-30472-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115480851507-30472-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115480851507-30472-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115480851507-30472-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.29.194.20:35045/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.194.2:46047
--local_ip_for_outbound_sockets=127.29.194.2
--tserver_master_addrs=127.29.194.62:41901
--webserver_port=36699
--webserver_interface=127.29.194.2
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.2.keytab
--principal=kudu/127.29.194.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.194.20:40769
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.29.194.20:35045/kms with env {KRB5CCNAME=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115480851507-30472-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115480851507-30472-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115480851507-30472-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 15:14:02 dist-test-slave-42z9 krb5kdc[15367](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115642, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.194.1@KRBTEST.COM for kudu/127.29.194.62@KRBTEST.COM
I20260807 15:14:02.186290 17236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:02.170233 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.194.1:58401 (local address 127.29.194.62:41901)
0807 15:14:02.170450 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0807 15:14:02.170459 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 15:14:02.170497 (+ 38us) server_negotiation.cc:407] Connection header received
0807 15:14:02.170959 (+ 462us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:14:02.170966 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:14:02.171017 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:14:02.171131 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:14:02.171763 (+ 632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:02.173000 (+ 1237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:02.174443 (+ 1443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:02.175053 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:02.175163 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:02.181922 (+ 6759us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:14:02.181929 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:14:02.181930 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:14:02.181944 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:14:02.184053 (+ 2109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:14:02.184613 (+ 560us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:14:02.184617 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:14:02.184617 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:14:02.184737 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:14:02.185045 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:14:02.185046 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:14:02.185046 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:14:02.185399 (+ 353us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:14:02.185490 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0807 15:14:02.185920 (+ 430us) server_negotiation.cc:299] Negotiation successful
0807 15:14:02.186064 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":174,"thread_start_us":84,"threads_started":1}
I20260807 15:14:02.187424 17235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:02.169936 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:41901 (local address 127.29.194.1:58401)
0807 15:14:02.170134 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0807 15:14:02.170178 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0807 15:14:02.170737 (+ 559us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:14:02.171185 (+ 448us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:14:02.171189 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:14:02.171419 (+ 230us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:14:02.171637 (+ 218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:14:02.171641 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:14:02.173131 (+ 1490us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:14:02.173133 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:14:02.174325 (+ 1192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:14:02.174329 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:14:02.175284 (+ 955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:14:02.175285 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:14:02.175437 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:02.180113 (+ 4676us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:14:02.180127 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:14:02.181743 (+ 1616us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:14:02.184168 (+ 2425us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:14:02.184177 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:14:02.184180 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:14:02.184507 (+ 327us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:14:02.184819 (+ 312us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:14:02.184820 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:14:02.184820 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:14:02.184961 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:14:02.185547 (+ 586us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:14:02.185554 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:14:02.185722 (+ 168us) client_negotiation.cc:769] Sending connection context
0807 15:14:02.187009 (+ 1287us) client_negotiation.cc:240] Negotiation successful
0807 15:14:02.187189 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":105,"threads_started":1}
I20260807 15:14:02.188508 17233 heartbeater.cc:347] Connected to a master server at 127.29.194.62:41901
I20260807 15:14:02.188604 17233 heartbeater.cc:464] Registering TS with master...
I20260807 15:14:02.188796 17233 heartbeater.cc:511] Master 127.29.194.62:41901 requested a full tablet report, sending...
W20260807 15:14:02.189720 17233 heartbeater.cc:644] Failed to heartbeat to 127.29.194.62:41901 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260807 15:14:02.189788 17233 heartbeater.cc:464] Registering TS with master...
I20260807 15:14:02.189848 17233 heartbeater.cc:511] Master 127.29.194.62:41901 requested a full tablet report, sending...
I20260807 15:14:02.190233 17233 heartbeater.cc:464] Registering TS with master...
I20260807 15:14:02.190304 17233 heartbeater.cc:511] Master 127.29.194.62:41901 requested a full tablet report, sending...
W20260807 15:14:02.190619 17233 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260807 15:14:02.285188 17238 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:14:02.285526 17238 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:14:02.285567 17238 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:14:02.287593 17238 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:14:02.287657 17238 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:14:02.287698 17238 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.194.2
I20260807 15:14:02.289541 17238 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.194.20:40769
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115480851507-30472-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115480851507-30472-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.29.194.20:35045/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.2.keytab
--principal=kudu/127.29.194.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.194.2:46047
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115480851507-30472-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.194.2
--webserver_port=36699
--tserver_master_addrs=127.29.194.62:41901
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17238
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.194.2
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115480851507-30472-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:14:02.289814 17238 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:14:02.290058 17238 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:14:02.295882 17248 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:14:02.295928 17252 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:14:02.296144 17249 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:14:02.296442 17238 server_base.cc:1034] running on GCE node
I20260807 15:14:02.296661 17238 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:14:02.297000 17238 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:14:02.298182 17238 hybrid_clock.cc:648] HybridClock initialized: now 1786115642298161 us; error 34 us; skew 500 ppm
Aug 07 15:14:02 dist-test-slave-42z9 krb5kdc[15367](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115642, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.194.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:14:02.301661 17238 init.cc:377] Logged in from keytab as kudu/127.29.194.2@KRBTEST.COM (short username kudu)
I20260807 15:14:02.302312 17238 webserver.cc:460] Webserver started at http://127.29.194.2:36699/ using document root <none> and password file <none>
I20260807 15:14:02.302650 17238 fs_manager.cc:359] Metadata directory not provided
I20260807 15:14:02.302747 17238 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 07 15:14:02 dist-test-slave-42z9 krb5kdc[15367](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115642, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.194.2@KRBTEST.COM for HTTP/127.29.194.20@KRBTEST.COM
I20260807 15:14:02.327500 17238 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:14:02.333962 17267 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:14:02.334261 17238 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.000s sys 0.000s
I20260807 15:14:02.334385 17238 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115480851507-30472-0/minicluster-data/ts-1/data,/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115480851507-30472-0/minicluster-data/ts-1/wal
uuid: "4709a7d453c74a9f88670579db66b8b8"
format_stamp: "Formatted at 2026-08-07 15:14:01 on dist-test-slave-42z9"
server_key: "da2050cb30663bc65f595ff6617e67b8"
server_key_iv: "af7053b2ca414edbd1156f670ee553f0"
server_key_version: "kuduclusterkey@0"
I20260807 15:14:02.334890 17238 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115480851507-30472-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115480851507-30472-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115480851507-30472-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:14:02.360672 17238 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:14:02.362906 17238 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:14:02.363063 17238 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:14:02.363309 17238 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:14:02.363675 17238 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:14:02.363713 17238 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:14:02.363744 17238 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:14:02.363793 17238 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:14:02.371307 17238 rpc_server.cc:307] RPC server started. Bound to: 127.29.194.2:46047
I20260807 15:14:02.371385 17465 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.194.2:46047 every 8 connection(s)
I20260807 15:14:02.371826 17238 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115480851507-30472-0/minicluster-data/ts-1/data/info.pb
I20260807 15:14:02.372171 30472 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7fZSIL/build/release/bin/kudu as pid 17238
I20260807 15:14:02.372463 30472 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7fZSIL/build/release/bin/kudu
/tmp/dist-test-task7fZSIL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115480851507-30472-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115480851507-30472-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115480851507-30472-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115480851507-30472-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.29.194.20:35045/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.194.3:45165
--local_ip_for_outbound_sockets=127.29.194.3
--tserver_master_addrs=127.29.194.62:41901
--webserver_port=34705
--webserver_interface=127.29.194.3
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.3.keytab
--principal=kudu/127.29.194.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.194.20:40769
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.29.194.20:35045/kms with env {KRB5CCNAME=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115480851507-30472-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115480851507-30472-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115480851507-30472-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 15:14:02 dist-test-slave-42z9 krb5kdc[15367](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115642, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.194.2@KRBTEST.COM for kudu/127.29.194.62@KRBTEST.COM
I20260807 15:14:02.381704 17236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:02.372671 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.194.2:39577 (local address 127.29.194.62:41901)
0807 15:14:02.372722 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0807 15:14:02.372724 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:14:02.372985 (+ 261us) server_negotiation.cc:407] Connection header received
0807 15:14:02.373532 (+ 547us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:14:02.373534 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:14:02.373547 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:14:02.373598 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:14:02.374169 (+ 571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:02.374929 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:02.376371 (+ 1442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:02.376586 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:02.376644 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:02.378821 (+ 2177us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:14:02.378823 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:14:02.378823 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:14:02.378831 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:14:02.380166 (+ 1335us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:14:02.380615 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:14:02.380616 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:14:02.380617 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:14:02.380682 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:14:02.380937 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:14:02.380938 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:14:02.380939 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:14:02.381090 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:14:02.381146 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0807 15:14:02.381507 (+ 361us) server_negotiation.cc:299] Negotiation successful
0807 15:14:02.381607 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260807 15:14:02.382052 17469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:02.372675 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:41901 (local address 127.29.194.2:39577)
0807 15:14:02.372837 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0807 15:14:02.372879 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0807 15:14:02.373381 (+ 502us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:14:02.373675 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:14:02.373679 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:14:02.373871 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:14:02.374077 (+ 206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:14:02.374081 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:14:02.375050 (+ 969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:14:02.375052 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:14:02.376275 (+ 1223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:14:02.376279 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:14:02.376690 (+ 411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:14:02.376691 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:14:02.376816 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:02.377331 (+ 515us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:14:02.377339 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:14:02.378723 (+ 1384us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:14:02.380270 (+ 1547us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:14:02.380279 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:14:02.380282 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:14:02.380541 (+ 259us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:14:02.380737 (+ 196us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:14:02.380737 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:14:02.380738 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:14:02.380864 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:14:02.381157 (+ 293us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:14:02.381166 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:14:02.381339 (+ 173us) client_negotiation.cc:769] Sending connection context
0807 15:14:02.381434 (+ 95us) client_negotiation.cc:240] Negotiation successful
0807 15:14:02.381628 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":91,"threads_started":1}
I20260807 15:14:02.382503 17467 heartbeater.cc:347] Connected to a master server at 127.29.194.62:41901
I20260807 15:14:02.382606 17467 heartbeater.cc:464] Registering TS with master...
I20260807 15:14:02.382797 17467 heartbeater.cc:511] Master 127.29.194.62:41901 requested a full tablet report, sending...
W20260807 15:14:02.383425 17467 heartbeater.cc:644] Failed to heartbeat to 127.29.194.62:41901 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260807 15:14:02.383502 17467 heartbeater.cc:464] Registering TS with master...
I20260807 15:14:02.383574 17467 heartbeater.cc:511] Master 127.29.194.62:41901 requested a full tablet report, sending...
I20260807 15:14:02.383863 17467 heartbeater.cc:464] Registering TS with master...
I20260807 15:14:02.383937 17467 heartbeater.cc:511] Master 127.29.194.62:41901 requested a full tablet report, sending...
W20260807 15:14:02.384202 17467 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260807 15:14:02.486313 17470 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:14:02.486641 17470 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:14:02.486688 17470 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:14:02.488615 17470 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:14:02.488695 17470 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:14:02.488739 17470 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.194.3
I20260807 15:14:02.490986 17470 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.194.20:40769
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115480851507-30472-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115480851507-30472-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.29.194.20:35045/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.3.keytab
--principal=kudu/127.29.194.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.194.3:45165
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115480851507-30472-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.194.3
--webserver_port=34705
--tserver_master_addrs=127.29.194.62:41901
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17470
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.194.3
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115480851507-30472-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:14:02.491272 17470 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:14:02.491521 17470 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:14:02.496739 17483 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:14:02.496773 17485 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:14:02.497179 17470 server_base.cc:1034] running on GCE node
W20260807 15:14:02.496829 17482 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:14:02.497612 17470 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:14:02.497948 17470 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:14:02.499150 17470 hybrid_clock.cc:648] HybridClock initialized: now 1786115642499132 us; error 36 us; skew 500 ppm
Aug 07 15:14:02 dist-test-slave-42z9 krb5kdc[15367](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115642, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.194.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:14:02.502039 17470 init.cc:377] Logged in from keytab as kudu/127.29.194.3@KRBTEST.COM (short username kudu)
I20260807 15:14:02.502513 17470 webserver.cc:460] Webserver started at http://127.29.194.3:34705/ using document root <none> and password file <none>
I20260807 15:14:02.502761 17470 fs_manager.cc:359] Metadata directory not provided
I20260807 15:14:02.502830 17470 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 07 15:14:02 dist-test-slave-42z9 krb5kdc[15367](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115642, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.194.3@KRBTEST.COM for HTTP/127.29.194.20@KRBTEST.COM
I20260807 15:14:02.530488 17470 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:14:02.534003 17497 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:14:02.534222 17470 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260807 15:14:02.534320 17470 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115480851507-30472-0/minicluster-data/ts-2/data,/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115480851507-30472-0/minicluster-data/ts-2/wal
uuid: "390340d1b4064577acddca506f6732d4"
format_stamp: "Formatted at 2026-08-07 15:14:01 on dist-test-slave-42z9"
server_key: "29f256276d753804cc09cc11bccd0400"
server_key_iv: "7e49721e977878c84091fcac3316b51b"
server_key_version: "kuduclusterkey@0"
I20260807 15:14:02.534852 17470 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115480851507-30472-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115480851507-30472-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115480851507-30472-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:14:02.573772 17470 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:14:02.575785 17470 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:14:02.575917 17470 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:14:02.576180 17470 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:14:02.576505 17470 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:14:02.576548 17470 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:14:02.576572 17470 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:14:02.576583 17470 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:14:02.582836 17470 rpc_server.cc:307] RPC server started. Bound to: 127.29.194.3:45165
I20260807 15:14:02.582964 17669 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.194.3:45165 every 8 connection(s)
I20260807 15:14:02.583346 17470 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115480851507-30472-0/minicluster-data/ts-2/data/info.pb
I20260807 15:14:02.591532 30472 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7fZSIL/build/release/bin/kudu as pid 17470
Aug 07 15:14:02 dist-test-slave-42z9 krb5kdc[15367](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115642, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.194.3@KRBTEST.COM for kudu/127.29.194.62@KRBTEST.COM
I20260807 15:14:02.597432 17236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:02.584169 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.194.3:50389 (local address 127.29.194.62:41901)
0807 15:14:02.584238 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0807 15:14:02.584240 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:14:02.586214 (+ 1974us) server_negotiation.cc:407] Connection header received
0807 15:14:02.586932 (+ 718us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:14:02.586934 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:14:02.586949 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:14:02.586982 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:14:02.587951 (+ 969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:02.588679 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:02.590431 (+ 1752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:02.590715 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:02.590764 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:02.593973 (+ 3209us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:14:02.593975 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:14:02.593976 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:14:02.593984 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:14:02.595732 (+ 1748us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:14:02.596290 (+ 558us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:14:02.596291 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:14:02.596292 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:14:02.596356 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:14:02.596670 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:14:02.596671 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:14:02.596671 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:14:02.596811 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:14:02.596861 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0807 15:14:02.597241 (+ 380us) server_negotiation.cc:299] Negotiation successful
0807 15:14:02.597329 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260807 15:14:02.598090 17672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:02.584151 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:41901 (local address 127.29.194.3:50389)
0807 15:14:02.586040 (+ 1889us) negotiation.cc:107] Waiting for socket to connect
0807 15:14:02.586101 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0807 15:14:02.586794 (+ 693us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:14:02.587253 (+ 459us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:14:02.587258 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:14:02.587526 (+ 268us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:14:02.587832 (+ 306us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:14:02.587839 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:14:02.588803 (+ 964us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:14:02.588804 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:14:02.590318 (+ 1514us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:14:02.590321 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:14:02.590898 (+ 577us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:14:02.590899 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:14:02.591059 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:02.592105 (+ 1046us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:14:02.592116 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:14:02.593849 (+ 1733us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:14:02.595844 (+ 1995us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:14:02.595852 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:14:02.595855 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:14:02.596190 (+ 335us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:14:02.596425 (+ 235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:14:02.596426 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:14:02.596426 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:14:02.596588 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:14:02.596896 (+ 308us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:14:02.596903 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:14:02.597082 (+ 179us) client_negotiation.cc:769] Sending connection context
0807 15:14:02.597635 (+ 553us) client_negotiation.cc:240] Negotiation successful
0807 15:14:02.597843 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1826,"thread_start_us":87,"threads_started":1}
I20260807 15:14:02.598619 17670 heartbeater.cc:347] Connected to a master server at 127.29.194.62:41901
I20260807 15:14:02.598726 17670 heartbeater.cc:464] Registering TS with master...
I20260807 15:14:02.598932 17670 heartbeater.cc:511] Master 127.29.194.62:41901 requested a full tablet report, sending...
W20260807 15:14:02.599805 17670 heartbeater.cc:644] Failed to heartbeat to 127.29.194.62:41901 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260807 15:14:02.599869 17670 heartbeater.cc:464] Registering TS with master...
I20260807 15:14:02.599921 17670 heartbeater.cc:511] Master 127.29.194.62:41901 requested a full tablet report, sending...
I20260807 15:14:02.600296 17670 heartbeater.cc:464] Registering TS with master...
I20260807 15:14:02.600363 17670 heartbeater.cc:511] Master 127.29.194.62:41901 requested a full tablet report, sending...
W20260807 15:14:02.600658 17670 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260807 15:14:02.603408 17673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:02.591951 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42948 (local address 127.29.194.62:41901)
0807 15:14:02.594328 (+ 2377us) server_negotiation.cc:206] Beginning negotiation
0807 15:14:02.594330 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:14:02.594857 (+ 527us) server_negotiation.cc:407] Connection header received
0807 15:14:02.594957 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:14:02.594958 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:14:02.594972 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:14:02.595042 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:14:02.595822 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:02.596590 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:02.597968 (+ 1378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:02.598648 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:02.598812 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:02.601007 (+ 2195us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:14:02.601008 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:14:02.601008 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:14:02.601014 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:14:02.602351 (+ 1337us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:14:02.602734 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:14:02.602735 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:14:02.602735 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:14:02.602784 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:14:02.602968 (+ 184us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:14:02.602969 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:14:02.602969 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:14:02.603066 (+ 97us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:14:02.603177 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0807 15:14:02.603264 (+ 87us) server_negotiation.cc:299] Negotiation successful
0807 15:14:02.603331 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2343,"thread_start_us":82,"threads_started":1}
Aug 07 15:14:03 dist-test-slave-42z9 krb5kdc[15367](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786115643, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.194.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:14:03.190850 17233 heartbeater.cc:464] Registering TS with master...
I20260807 15:14:03.191025 17233 heartbeater.cc:511] Master 127.29.194.62:41901 requested a full tablet report, sending...
I20260807 15:14:03.191373 16934 ts_manager.cc:194] Registered new tserver with Master: de619175db0a4b49946a0f3a5cb7f5d7 (127.29.194.1:40479)
I20260807 15:14:03.192062 16934 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.194.1@KRBTEST.COM'} at 127.29.194.1:58401
I20260807 15:14:03.384374 17467 heartbeater.cc:464] Registering TS with master...
I20260807 15:14:03.384567 17467 heartbeater.cc:511] Master 127.29.194.62:41901 requested a full tablet report, sending...
I20260807 15:14:03.384891 16934 ts_manager.cc:194] Registered new tserver with Master: 4709a7d453c74a9f88670579db66b8b8 (127.29.194.2:46047)
I20260807 15:14:03.385416 16934 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.194.2@KRBTEST.COM'} at 127.29.194.2:39577
Aug 07 15:14:03 dist-test-slave-42z9 krb5kdc[15367](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115643, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.194.62@KRBTEST.COM for HTTP/127.29.194.20@KRBTEST.COM
I20260807 15:14:03.600868 17670 heartbeater.cc:464] Registering TS with master...
I20260807 15:14:03.601050 17670 heartbeater.cc:511] Master 127.29.194.62:41901 requested a full tablet report, sending...
I20260807 15:14:03.601327 16934 ts_manager.cc:194] Registered new tserver with Master: 390340d1b4064577acddca506f6732d4 (127.29.194.3:45165)
I20260807 15:14:03.601886 16934 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.194.3@KRBTEST.COM'} at 127.29.194.3:50389
I20260807 15:14:03.612485 30472 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 15:14:03.621388 17698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:03.613574 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42964 (local address 127.29.194.62:41901)
0807 15:14:03.613771 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0807 15:14:03.613773 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:14:03.613781 (+ 8us) server_negotiation.cc:407] Connection header received
0807 15:14:03.613804 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:14:03.613805 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:14:03.613817 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:14:03.613897 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:14:03.614711 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:03.615452 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:03.616324 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:03.616649 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:03.616729 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:03.618139 (+ 1410us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:14:03.618140 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:14:03.618140 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:14:03.618149 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:14:03.620123 (+ 1974us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:14:03.620533 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:14:03.620534 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:14:03.620535 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:14:03.620607 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:14:03.620764 (+ 157us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:14:03.620765 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:14:03.620765 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:14:03.620911 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:14:03.621022 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0807 15:14:03.621181 (+ 159us) server_negotiation.cc:299] Negotiation successful
0807 15:14:03.621276 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":172,"thread_start_us":126,"threads_started":1}
I20260807 15:14:03.623962 16934 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42964:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 15:14:03.626773 16934 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 15:14:03.635457 17716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:03.630244 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.2:46047 (local address 127.0.0.1:56176)
0807 15:14:03.630846 (+ 602us) negotiation.cc:107] Waiting for socket to connect
0807 15:14:03.630889 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0807 15:14:03.630960 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:14:03.631735 (+ 775us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:14:03.631736 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:14:03.631739 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:14:03.631795 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:14:03.631797 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:14:03.632979 (+ 1182us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:14:03.632980 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:14:03.634128 (+ 1148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:14:03.634133 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:14:03.635139 (+ 1006us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:14:03.635140 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:14:03.635227 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:03.635241 (+ 14us) client_negotiation.cc:769] Sending connection context
0807 15:14:03.635302 (+ 61us) client_negotiation.cc:240] Negotiation successful
0807 15:14:03.635372 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":571,"mutex_wait_us":2,"thread_start_us":78,"threads_started":1}
I20260807 15:14:03.635501 17714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:03.630582 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58222 (local address 127.29.194.3:45165)
0807 15:14:03.630729 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0807 15:14:03.630734 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:14:03.630954 (+ 220us) server_negotiation.cc:407] Connection header received
0807 15:14:03.631067 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:14:03.631069 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:14:03.631086 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:14:03.631138 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:14:03.631431 (+ 293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:03.632690 (+ 1259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:03.634211 (+ 1521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:03.635016 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:03.635086 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:03.635146 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0807 15:14:03.635366 (+ 220us) server_negotiation.cc:299] Negotiation successful
0807 15:14:03.635412 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":118,"thread_start_us":60,"threads_started":1}
I20260807 15:14:03.635725 17713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:03.630501 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.3:45165 (local address 127.0.0.1:58222)
0807 15:14:03.630847 (+ 346us) negotiation.cc:107] Waiting for socket to connect
0807 15:14:03.630889 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0807 15:14:03.630988 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:14:03.631261 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:14:03.631264 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:14:03.631267 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:14:03.631337 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:14:03.631339 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:14:03.632800 (+ 1461us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:14:03.632801 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:14:03.634108 (+ 1307us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:14:03.634112 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:14:03.635184 (+ 1072us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:14:03.635185 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:14:03.635251 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:03.635256 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 15:14:03.635315 (+ 59us) client_negotiation.cc:240] Negotiation successful
0807 15:14:03.635654 (+ 339us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":319,"mutex_wait_us":283,"thread_start_us":81,"threads_started":1}
I20260807 15:14:03.636225 17712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:03.630333 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56176 (local address 127.29.194.2:46047)
0807 15:14:03.631489 (+ 1156us) server_negotiation.cc:206] Beginning negotiation
0807 15:14:03.631494 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:14:03.631505 (+ 11us) server_negotiation.cc:407] Connection header received
0807 15:14:03.631542 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:14:03.631545 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:14:03.631561 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:14:03.631637 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:14:03.631873 (+ 236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:03.632880 (+ 1007us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:03.634230 (+ 1350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:03.634976 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:03.635042 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:03.635104 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0807 15:14:03.636097 (+ 993us) server_negotiation.cc:299] Negotiation successful
0807 15:14:03.636147 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1105,"thread_start_us":71,"threads_started":1}
I20260807 15:14:03.636854 17555 tablet_service.cc:1467] Processing CreateTablet for tablet ce871db4c82b46f08cef73ad6d81182e (DEFAULT_TABLE table=test-table [id=36aefd6f1aae45bea4e079c3b98c44dd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:14:03.636952 17353 tablet_service.cc:1467] Processing CreateTablet for tablet ce871db4c82b46f08cef73ad6d81182e (DEFAULT_TABLE table=test-table [id=36aefd6f1aae45bea4e079c3b98c44dd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:14:03.637285 17353 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ce871db4c82b46f08cef73ad6d81182e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:14:03.637285 17555 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ce871db4c82b46f08cef73ad6d81182e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:14:03.640972 17726 tablet_bootstrap.cc:492] T ce871db4c82b46f08cef73ad6d81182e P 390340d1b4064577acddca506f6732d4: Bootstrap starting.
I20260807 15:14:03.642038 17726 tablet_bootstrap.cc:654] T ce871db4c82b46f08cef73ad6d81182e P 390340d1b4064577acddca506f6732d4: Neither blocks nor log segments found. Creating new log.
I20260807 15:14:03.642362 17726 log.cc:824] T ce871db4c82b46f08cef73ad6d81182e P 390340d1b4064577acddca506f6732d4: Log is configured to *not* fsync() on all Append() calls
I20260807 15:14:03.642755 17725 tablet_bootstrap.cc:492] T ce871db4c82b46f08cef73ad6d81182e P 4709a7d453c74a9f88670579db66b8b8: Bootstrap starting.
I20260807 15:14:03.643344 17726 tablet_bootstrap.cc:492] T ce871db4c82b46f08cef73ad6d81182e P 390340d1b4064577acddca506f6732d4: No bootstrap required, opened a new log
I20260807 15:14:03.643420 17726 ts_tablet_manager.cc:1397] T ce871db4c82b46f08cef73ad6d81182e P 390340d1b4064577acddca506f6732d4: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260807 15:14:03.646114 17726 raft_consensus.cc:348] T ce871db4c82b46f08cef73ad6d81182e P 390340d1b4064577acddca506f6732d4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4709a7d453c74a9f88670579db66b8b8" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 46047 } } peers { permanent_uuid: "390340d1b4064577acddca506f6732d4" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 45165 } } peers { permanent_uuid: "de619175db0a4b49946a0f3a5cb7f5d7" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 40479 } }
I20260807 15:14:03.646314 17726 raft_consensus.cc:374] T ce871db4c82b46f08cef73ad6d81182e P 390340d1b4064577acddca506f6732d4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:14:03.646361 17726 raft_consensus.cc:729] T ce871db4c82b46f08cef73ad6d81182e P 390340d1b4064577acddca506f6732d4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 390340d1b4064577acddca506f6732d4, State: Initialized, Role: FOLLOWER
I20260807 15:14:03.646512 17726 consensus_queue.cc:260] T ce871db4c82b46f08cef73ad6d81182e P 390340d1b4064577acddca506f6732d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4709a7d453c74a9f88670579db66b8b8" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 46047 } } peers { permanent_uuid: "390340d1b4064577acddca506f6732d4" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 45165 } } peers { permanent_uuid: "de619175db0a4b49946a0f3a5cb7f5d7" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 40479 } }
I20260807 15:14:03.646847 17670 heartbeater.cc:503] Master 127.29.194.62:41901 was elected leader, sending a full tablet report...
I20260807 15:14:03.647153 17726 ts_tablet_manager.cc:1428] T ce871db4c82b46f08cef73ad6d81182e P 390340d1b4064577acddca506f6732d4: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260807 15:14:03.648197 17725 tablet_bootstrap.cc:654] T ce871db4c82b46f08cef73ad6d81182e P 4709a7d453c74a9f88670579db66b8b8: Neither blocks nor log segments found. Creating new log.
I20260807 15:14:03.648550 17725 log.cc:824] T ce871db4c82b46f08cef73ad6d81182e P 4709a7d453c74a9f88670579db66b8b8: Log is configured to *not* fsync() on all Append() calls
I20260807 15:14:03.649521 17725 tablet_bootstrap.cc:492] T ce871db4c82b46f08cef73ad6d81182e P 4709a7d453c74a9f88670579db66b8b8: No bootstrap required, opened a new log
I20260807 15:14:03.649603 17725 ts_tablet_manager.cc:1397] T ce871db4c82b46f08cef73ad6d81182e P 4709a7d453c74a9f88670579db66b8b8: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20260807 15:14:03.652421 17725 raft_consensus.cc:348] T ce871db4c82b46f08cef73ad6d81182e P 4709a7d453c74a9f88670579db66b8b8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4709a7d453c74a9f88670579db66b8b8" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 46047 } } peers { permanent_uuid: "390340d1b4064577acddca506f6732d4" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 45165 } } peers { permanent_uuid: "de619175db0a4b49946a0f3a5cb7f5d7" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 40479 } }
I20260807 15:14:03.652631 17725 raft_consensus.cc:374] T ce871db4c82b46f08cef73ad6d81182e P 4709a7d453c74a9f88670579db66b8b8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:14:03.652674 17725 raft_consensus.cc:729] T ce871db4c82b46f08cef73ad6d81182e P 4709a7d453c74a9f88670579db66b8b8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4709a7d453c74a9f88670579db66b8b8, State: Initialized, Role: FOLLOWER
I20260807 15:14:03.652842 17725 consensus_queue.cc:260] T ce871db4c82b46f08cef73ad6d81182e P 4709a7d453c74a9f88670579db66b8b8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4709a7d453c74a9f88670579db66b8b8" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 46047 } } peers { permanent_uuid: "390340d1b4064577acddca506f6732d4" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 45165 } } peers { permanent_uuid: "de619175db0a4b49946a0f3a5cb7f5d7" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 40479 } }
I20260807 15:14:03.653184 17725 ts_tablet_manager.cc:1428] T ce871db4c82b46f08cef73ad6d81182e P 4709a7d453c74a9f88670579db66b8b8: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260807 15:14:03.653327 17467 heartbeater.cc:503] Master 127.29.194.62:41901 was elected leader, sending a full tablet report...
I20260807 15:14:03.654292 17716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:03.630713 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.1:40479 (local address 127.0.0.1:33454)
0807 15:14:03.635986 (+ 5273us) negotiation.cc:107] Waiting for socket to connect
0807 15:14:03.635996 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 15:14:03.636028 (+ 32us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:14:03.649571 (+ 13543us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:14:03.649573 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:14:03.649576 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:14:03.649634 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:14:03.649635 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:14:03.650971 (+ 1336us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:14:03.650972 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:14:03.652272 (+ 1300us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:14:03.652282 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:14:03.654034 (+ 1752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:14:03.654035 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:14:03.654117 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:03.654122 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 15:14:03.654151 (+ 29us) client_negotiation.cc:240] Negotiation successful
0807 15:14:03.654215 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5260,"thread_start_us":71,"threads_started":1}
I20260807 15:14:03.654897 17720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:03.634020 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33454 (local address 127.29.194.1:40479)
0807 15:14:03.642867 (+ 8847us) server_negotiation.cc:206] Beginning negotiation
0807 15:14:03.642872 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:14:03.642887 (+ 15us) server_negotiation.cc:407] Connection header received
0807 15:14:03.642927 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:14:03.642929 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:14:03.642945 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:14:03.649776 (+ 6831us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:14:03.649868 (+ 92us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:03.650870 (+ 1002us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:03.653012 (+ 2142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:03.653916 (+ 904us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:03.654651 (+ 735us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:03.654735 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0807 15:14:03.654765 (+ 30us) server_negotiation.cc:299] Negotiation successful
0807 15:14:03.654814 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8802,"thread_start_us":76,"threads_started":1}
I20260807 15:14:03.655819 17137 tablet_service.cc:1467] Processing CreateTablet for tablet ce871db4c82b46f08cef73ad6d81182e (DEFAULT_TABLE table=test-table [id=36aefd6f1aae45bea4e079c3b98c44dd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:14:03.656214 17137 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ce871db4c82b46f08cef73ad6d81182e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:14:03.659895 17731 tablet_bootstrap.cc:492] T ce871db4c82b46f08cef73ad6d81182e P de619175db0a4b49946a0f3a5cb7f5d7: Bootstrap starting.
I20260807 15:14:03.660755 17731 tablet_bootstrap.cc:654] T ce871db4c82b46f08cef73ad6d81182e P de619175db0a4b49946a0f3a5cb7f5d7: Neither blocks nor log segments found. Creating new log.
I20260807 15:14:03.661178 17731 log.cc:824] T ce871db4c82b46f08cef73ad6d81182e P de619175db0a4b49946a0f3a5cb7f5d7: Log is configured to *not* fsync() on all Append() calls
I20260807 15:14:03.662125 17731 tablet_bootstrap.cc:492] T ce871db4c82b46f08cef73ad6d81182e P de619175db0a4b49946a0f3a5cb7f5d7: No bootstrap required, opened a new log
I20260807 15:14:03.662206 17731 ts_tablet_manager.cc:1397] T ce871db4c82b46f08cef73ad6d81182e P de619175db0a4b49946a0f3a5cb7f5d7: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260807 15:14:03.664948 17731 raft_consensus.cc:348] T ce871db4c82b46f08cef73ad6d81182e P de619175db0a4b49946a0f3a5cb7f5d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4709a7d453c74a9f88670579db66b8b8" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 46047 } } peers { permanent_uuid: "390340d1b4064577acddca506f6732d4" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 45165 } } peers { permanent_uuid: "de619175db0a4b49946a0f3a5cb7f5d7" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 40479 } }
I20260807 15:14:03.665175 17731 raft_consensus.cc:374] T ce871db4c82b46f08cef73ad6d81182e P de619175db0a4b49946a0f3a5cb7f5d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:14:03.665215 17731 raft_consensus.cc:729] T ce871db4c82b46f08cef73ad6d81182e P de619175db0a4b49946a0f3a5cb7f5d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de619175db0a4b49946a0f3a5cb7f5d7, State: Initialized, Role: FOLLOWER
I20260807 15:14:03.665378 17731 consensus_queue.cc:260] T ce871db4c82b46f08cef73ad6d81182e P de619175db0a4b49946a0f3a5cb7f5d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4709a7d453c74a9f88670579db66b8b8" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 46047 } } peers { permanent_uuid: "390340d1b4064577acddca506f6732d4" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 45165 } } peers { permanent_uuid: "de619175db0a4b49946a0f3a5cb7f5d7" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 40479 } }
I20260807 15:14:03.665743 17731 ts_tablet_manager.cc:1428] T ce871db4c82b46f08cef73ad6d81182e P de619175db0a4b49946a0f3a5cb7f5d7: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260807 15:14:03.665894 17233 heartbeater.cc:503] Master 127.29.194.62:41901 was elected leader, sending a full tablet report...
W20260807 15:14:03.670420 17234 tablet.cc:2369] T ce871db4c82b46f08cef73ad6d81182e P de619175db0a4b49946a0f3a5cb7f5d7: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 15:14:03.699863 17728 raft_consensus.cc:482] T ce871db4c82b46f08cef73ad6d81182e P 390340d1b4064577acddca506f6732d4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 15:14:03.700033 17728 raft_consensus.cc:504] T ce871db4c82b46f08cef73ad6d81182e P 390340d1b4064577acddca506f6732d4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4709a7d453c74a9f88670579db66b8b8" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 46047 } } peers { permanent_uuid: "390340d1b4064577acddca506f6732d4" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 45165 } } peers { permanent_uuid: "de619175db0a4b49946a0f3a5cb7f5d7" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 40479 } }
I20260807 15:14:03.700475 17728 leader_election.cc:288] T ce871db4c82b46f08cef73ad6d81182e P 390340d1b4064577acddca506f6732d4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4709a7d453c74a9f88670579db66b8b8 (127.29.194.2:46047), de619175db0a4b49946a0f3a5cb7f5d7 (127.29.194.1:40479)
I20260807 15:14:03.704802 17712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:03.700932 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.194.3:56217 (local address 127.29.194.2:46047)
0807 15:14:03.700981 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0807 15:14:03.700983 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:14:03.701117 (+ 134us) server_negotiation.cc:407] Connection header received
0807 15:14:03.701220 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:14:03.701222 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:14:03.701231 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:14:03.701258 (+ 27us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:14:03.701555 (+ 297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:03.702264 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:03.703837 (+ 1573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:03.704389 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:03.704437 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:03.704447 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0807 15:14:03.704676 (+ 229us) server_negotiation.cc:299] Negotiation successful
0807 15:14:03.704725 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260807 15:14:03.705096 17738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:03.700661 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.2:46047 (local address 127.29.194.3:56217)
0807 15:14:03.701043 (+ 382us) negotiation.cc:107] Waiting for socket to connect
0807 15:14:03.701054 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 15:14:03.701148 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:14:03.701334 (+ 186us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:14:03.701334 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:14:03.701336 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:14:03.701480 (+ 144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:14:03.701482 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:14:03.702378 (+ 896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:14:03.702379 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:14:03.703719 (+ 1340us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:14:03.703723 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:14:03.704523 (+ 800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:14:03.704524 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:14:03.704592 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:03.704596 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 15:14:03.704947 (+ 351us) client_negotiation.cc:240] Negotiation successful
0807 15:14:03.704999 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":364,"thread_start_us":128,"threads_started":1}
I20260807 15:14:03.705477 17385 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ce871db4c82b46f08cef73ad6d81182e" candidate_uuid: "390340d1b4064577acddca506f6732d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4709a7d453c74a9f88670579db66b8b8" is_pre_election: true
I20260807 15:14:03.705561 17720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:03.700941 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.194.3:50411 (local address 127.29.194.1:40479)
0807 15:14:03.700985 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0807 15:14:03.700987 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:14:03.701195 (+ 208us) server_negotiation.cc:407] Connection header received
0807 15:14:03.701293 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:14:03.701294 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:14:03.701304 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:14:03.701330 (+ 26us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:14:03.701574 (+ 244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:03.702330 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:03.704158 (+ 1828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:03.704728 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:03.704773 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:03.704780 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0807 15:14:03.705414 (+ 634us) server_negotiation.cc:299] Negotiation successful
0807 15:14:03.705491 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260807 15:14:03.705652 17385 raft_consensus.cc:2454] T ce871db4c82b46f08cef73ad6d81182e P 4709a7d453c74a9f88670579db66b8b8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 390340d1b4064577acddca506f6732d4 in term 0.
I20260807 15:14:03.705880 17501 leader_election.cc:302] T ce871db4c82b46f08cef73ad6d81182e P 390340d1b4064577acddca506f6732d4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 390340d1b4064577acddca506f6732d4, 4709a7d453c74a9f88670579db66b8b8; no voters:
I20260807 15:14:03.706022 17728 raft_consensus.cc:2781] T ce871db4c82b46f08cef73ad6d81182e P 390340d1b4064577acddca506f6732d4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 15:14:03.706089 17728 raft_consensus.cc:482] T ce871db4c82b46f08cef73ad6d81182e P 390340d1b4064577acddca506f6732d4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 15:14:03.706117 17728 raft_consensus.cc:3037] T ce871db4c82b46f08cef73ad6d81182e P 390340d1b4064577acddca506f6732d4 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:14:03.706193 17739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:03.700702 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.1:40479 (local address 127.29.194.3:50411)
0807 15:14:03.701125 (+ 423us) negotiation.cc:107] Waiting for socket to connect
0807 15:14:03.701136 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 15:14:03.701223 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:14:03.701391 (+ 168us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:14:03.701392 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:14:03.701394 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:14:03.701495 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:14:03.701497 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:14:03.702453 (+ 956us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:14:03.702454 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:14:03.704050 (+ 1596us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:14:03.704053 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:14:03.704835 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:14:03.704836 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:14:03.705315 (+ 479us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:03.705322 (+ 7us) client_negotiation.cc:769] Sending connection context
0807 15:14:03.705855 (+ 533us) client_negotiation.cc:240] Negotiation successful
0807 15:14:03.705906 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":397,"mutex_wait_us":425,"thread_start_us":105,"threads_started":1}
I20260807 15:14:03.706616 17154 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ce871db4c82b46f08cef73ad6d81182e" candidate_uuid: "390340d1b4064577acddca506f6732d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de619175db0a4b49946a0f3a5cb7f5d7" is_pre_election: true
I20260807 15:14:03.706769 17154 raft_consensus.cc:2454] T ce871db4c82b46f08cef73ad6d81182e P de619175db0a4b49946a0f3a5cb7f5d7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 390340d1b4064577acddca506f6732d4 in term 0.
I20260807 15:14:03.707016 17728 raft_consensus.cc:504] T ce871db4c82b46f08cef73ad6d81182e P 390340d1b4064577acddca506f6732d4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4709a7d453c74a9f88670579db66b8b8" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 46047 } } peers { permanent_uuid: "390340d1b4064577acddca506f6732d4" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 45165 } } peers { permanent_uuid: "de619175db0a4b49946a0f3a5cb7f5d7" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 40479 } }
I20260807 15:14:03.707149 17728 leader_election.cc:288] T ce871db4c82b46f08cef73ad6d81182e P 390340d1b4064577acddca506f6732d4 [CANDIDATE]: Term 1 election: Requested vote from peers 4709a7d453c74a9f88670579db66b8b8 (127.29.194.2:46047), de619175db0a4b49946a0f3a5cb7f5d7 (127.29.194.1:40479)
I20260807 15:14:03.707332 17154 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ce871db4c82b46f08cef73ad6d81182e" candidate_uuid: "390340d1b4064577acddca506f6732d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de619175db0a4b49946a0f3a5cb7f5d7"
I20260807 15:14:03.707331 17385 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ce871db4c82b46f08cef73ad6d81182e" candidate_uuid: "390340d1b4064577acddca506f6732d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4709a7d453c74a9f88670579db66b8b8"
I20260807 15:14:03.707407 17154 raft_consensus.cc:3037] T ce871db4c82b46f08cef73ad6d81182e P de619175db0a4b49946a0f3a5cb7f5d7 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:14:03.707410 17385 raft_consensus.cc:3037] T ce871db4c82b46f08cef73ad6d81182e P 4709a7d453c74a9f88670579db66b8b8 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:14:03.708323 17385 raft_consensus.cc:2454] T ce871db4c82b46f08cef73ad6d81182e P 4709a7d453c74a9f88670579db66b8b8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 390340d1b4064577acddca506f6732d4 in term 1.
I20260807 15:14:03.708356 17154 raft_consensus.cc:2454] T ce871db4c82b46f08cef73ad6d81182e P de619175db0a4b49946a0f3a5cb7f5d7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 390340d1b4064577acddca506f6732d4 in term 1.
I20260807 15:14:03.708492 17501 leader_election.cc:302] T ce871db4c82b46f08cef73ad6d81182e P 390340d1b4064577acddca506f6732d4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 390340d1b4064577acddca506f6732d4, 4709a7d453c74a9f88670579db66b8b8; no voters:
I20260807 15:14:03.708566 17728 raft_consensus.cc:2781] T ce871db4c82b46f08cef73ad6d81182e P 390340d1b4064577acddca506f6732d4 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:14:03.708716 17728 raft_consensus.cc:686] T ce871db4c82b46f08cef73ad6d81182e P 390340d1b4064577acddca506f6732d4 [term 1 LEADER]: Becoming Leader. State: Replica: 390340d1b4064577acddca506f6732d4, State: Running, Role: LEADER
I20260807 15:14:03.708820 17728 consensus_queue.cc:237] T ce871db4c82b46f08cef73ad6d81182e P 390340d1b4064577acddca506f6732d4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4709a7d453c74a9f88670579db66b8b8" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 46047 } } peers { permanent_uuid: "390340d1b4064577acddca506f6732d4" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 45165 } } peers { permanent_uuid: "de619175db0a4b49946a0f3a5cb7f5d7" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 40479 } }
I20260807 15:14:03.709733 16934 catalog_manager.cc:5495] T ce871db4c82b46f08cef73ad6d81182e P 390340d1b4064577acddca506f6732d4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 390340d1b4064577acddca506f6732d4 (127.29.194.3). New cstate: current_term: 1 leader_uuid: "390340d1b4064577acddca506f6732d4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4709a7d453c74a9f88670579db66b8b8" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 46047 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "390340d1b4064577acddca506f6732d4" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 45165 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "de619175db0a4b49946a0f3a5cb7f5d7" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 40479 } health_report { overall_health: UNKNOWN } } }
I20260807 15:14:03.723070 17714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:03.719921 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58236 (local address 127.29.194.3:45165)
0807 15:14:03.719998 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0807 15:14:03.720007 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 15:14:03.720019 (+ 12us) server_negotiation.cc:407] Connection header received
0807 15:14:03.720061 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:14:03.720062 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:14:03.720084 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:14:03.720171 (+ 87us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 15:14:03.720288 (+ 117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:03.721022 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:03.722061 (+ 1039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:03.722373 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:03.722452 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:03.722627 (+ 175us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 15:14:03.722780 (+ 153us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 15:14:03.722888 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0807 15:14:03.722908 (+ 20us) server_negotiation.cc:299] Negotiation successful
0807 15:14:03.722936 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260807 15:14:03.725392 17154 raft_consensus.cc:1261] T ce871db4c82b46f08cef73ad6d81182e P de619175db0a4b49946a0f3a5cb7f5d7 [term 1 FOLLOWER]: Refusing update from remote peer 390340d1b4064577acddca506f6732d4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:14:03.725428 17385 raft_consensus.cc:1261] T ce871db4c82b46f08cef73ad6d81182e P 4709a7d453c74a9f88670579db66b8b8 [term 1 FOLLOWER]: Refusing update from remote peer 390340d1b4064577acddca506f6732d4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:14:03.725832 17728 consensus_queue.cc:1035] T ce871db4c82b46f08cef73ad6d81182e P 390340d1b4064577acddca506f6732d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4709a7d453c74a9f88670579db66b8b8" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 46047 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:14:03.726137 17740 consensus_queue.cc:1035] T ce871db4c82b46f08cef73ad6d81182e P 390340d1b4064577acddca506f6732d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de619175db0a4b49946a0f3a5cb7f5d7" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 40479 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:14:03.730530 17748 mvcc.cc:205] Tried to move back new op lower bound from 7315929676696408064 to 7315929676631859200. Current Snapshot: MvccSnapshot[applied={T|T < 7315929676696408064}]
I20260807 15:14:03.733137 16933 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42964:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 15:14:03.733251 16933 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42964:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 15:14:03.734277 16933 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9739f9d5895d48248a63bcfa20d35884: Sending DeleteTablet for 3 replicas of tablet ce871db4c82b46f08cef73ad6d81182e
I20260807 15:14:03.734670 17137 tablet_service.cc:1514] Processing DeleteTablet for tablet ce871db4c82b46f08cef73ad6d81182e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:14:03 UTC) from {username='kudu', principal='kudu/127.29.194.62@KRBTEST.COM'} at 127.0.0.1:33454
I20260807 15:14:03.734655 17555 tablet_service.cc:1514] Processing DeleteTablet for tablet ce871db4c82b46f08cef73ad6d81182e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:14:03 UTC) from {username='kudu', principal='kudu/127.29.194.62@KRBTEST.COM'} at 127.0.0.1:58222
I20260807 15:14:03.734727 17353 tablet_service.cc:1514] Processing DeleteTablet for tablet ce871db4c82b46f08cef73ad6d81182e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:14:03 UTC) from {username='kudu', principal='kudu/127.29.194.62@KRBTEST.COM'} at 127.0.0.1:56176
I20260807 15:14:03.735355 30472 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7fZSIL/build/release/bin/kudu with pid 16989
I20260807 15:14:03.739921 17758 tablet_replica.cc:330] T ce871db4c82b46f08cef73ad6d81182e P 4709a7d453c74a9f88670579db66b8b8: stopping tablet replica
I20260807 15:14:03.740192 17758 raft_consensus.cc:2229] T ce871db4c82b46f08cef73ad6d81182e P 4709a7d453c74a9f88670579db66b8b8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 15:14:03.740398 17758 raft_consensus.cc:2258] T ce871db4c82b46f08cef73ad6d81182e P 4709a7d453c74a9f88670579db66b8b8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:14:03.749632 17756 tablet_replica.cc:330] T ce871db4c82b46f08cef73ad6d81182e P 390340d1b4064577acddca506f6732d4: stopping tablet replica
I20260807 15:14:03.749863 17756 raft_consensus.cc:2229] T ce871db4c82b46f08cef73ad6d81182e P 390340d1b4064577acddca506f6732d4 [term 1 LEADER]: Raft consensus shutting down.
I20260807 15:14:03.750041 17756 raft_consensus.cc:2258] T ce871db4c82b46f08cef73ad6d81182e P 390340d1b4064577acddca506f6732d4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:14:03.750581 17756 ts_tablet_manager.cc:1905] T ce871db4c82b46f08cef73ad6d81182e P 390340d1b4064577acddca506f6732d4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:14:03.750586 17758 ts_tablet_manager.cc:1905] T ce871db4c82b46f08cef73ad6d81182e P 4709a7d453c74a9f88670579db66b8b8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:14:03.752337 17756 ts_tablet_manager.cc:1918] T ce871db4c82b46f08cef73ad6d81182e P 390340d1b4064577acddca506f6732d4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 15:14:03.752434 17756 log.cc:1196] T ce871db4c82b46f08cef73ad6d81182e P 390340d1b4064577acddca506f6732d4: Deleting WAL directory at /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115480851507-30472-0/minicluster-data/ts-2/wal/wals/ce871db4c82b46f08cef73ad6d81182e
I20260807 15:14:03.752692 17756 ts_tablet_manager.cc:1939] T ce871db4c82b46f08cef73ad6d81182e P 390340d1b4064577acddca506f6732d4: Deleting consensus metadata
I20260807 15:14:03.753201 16913 catalog_manager.cc:4841] TS 390340d1b4064577acddca506f6732d4 (127.29.194.3:45165): tablet ce871db4c82b46f08cef73ad6d81182e (table test-table [id=36aefd6f1aae45bea4e079c3b98c44dd]) successfully deleted
I20260807 15:14:03.754299 17758 ts_tablet_manager.cc:1918] T ce871db4c82b46f08cef73ad6d81182e P 4709a7d453c74a9f88670579db66b8b8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 15:14:03.757774 17758 log.cc:1196] T ce871db4c82b46f08cef73ad6d81182e P 4709a7d453c74a9f88670579db66b8b8: Deleting WAL directory at /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115480851507-30472-0/minicluster-data/ts-1/wal/wals/ce871db4c82b46f08cef73ad6d81182e
I20260807 15:14:03.758234 17758 ts_tablet_manager.cc:1939] T ce871db4c82b46f08cef73ad6d81182e P 4709a7d453c74a9f88670579db66b8b8: Deleting consensus metadata
I20260807 15:14:03.758790 16916 catalog_manager.cc:4841] TS 4709a7d453c74a9f88670579db66b8b8 (127.29.194.2:46047): tablet ce871db4c82b46f08cef73ad6d81182e (table test-table [id=36aefd6f1aae45bea4e079c3b98c44dd]) successfully deleted
W20260807 15:14:03.759183 16915 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.29.194.1:40479 (error 108)
I20260807 15:14:03.759881 30472 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7fZSIL/build/release/bin/kudu with pid 17238
I20260807 15:14:03.815904 17716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:03.815714 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.1:40479 (local address 127.0.0.1:33470)
0807 15:14:03.815770 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0807 15:14:03.815852 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.194.1:40479: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
W20260807 15:14:03.817422 16915 catalog_manager.cc:4584] TS de619175db0a4b49946a0f3a5cb7f5d7 (127.29.194.1:40479): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet ce871db4c82b46f08cef73ad6d81182e: Network error: Client connection negotiation failed: client connection to 127.29.194.1:40479: connect: Connection refused (error 111)
I20260807 15:14:03.823103 30472 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7fZSIL/build/release/bin/kudu with pid 17470
I20260807 15:14:03.835240 30472 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7fZSIL/build/release/bin/kudu with pid 16884
I20260807 15:14:03.847143 30472 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260807 15:14:04.193274 30472 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260807 15:14:04.193392 30472 mini_ranger.cc:66] Stopping Ranger...
I20260807 15:14:04.538872 30472 mini_ranger.cc:68] Stopped Ranger
2026-08-07 15:14:04.539 UTC [15415] LOG: received smart shutdown request
2026-08-07 15:14:04.544 UTC [15415] LOG: background worker "logical replication launcher" (PID 15428) exited with exit code 1
2026-08-07 15:14:04.544 UTC [15422] LOG: shutting down
2026-08-07 15:14:04.724 UTC [15415] LOG: database system is shut down
2026-08-07T15:14:04Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (50470 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115480851507-30472-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 15:14:04 dist-test-slave-42z9 krb5kdc[17779](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 15:14:04 dist-test-slave-42z9 krb5kdc[17779](info): set up 2 sockets
Aug 07 15:14:04 dist-test-slave-42z9 krb5kdc[17779](info): commencing operation
krb5kdc: starting...
W20260807 15:14:05.960873 30472 mini_kdc.cc:121] Time spent starting KDC: real 1.119s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115480851507-30472-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115480851507-30472-0/krb5kdc/test-user.keytab.
Aug 07 15:14:06 dist-test-slave-42z9 krb5kdc[17779](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115646, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T15:14:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T15:14:06Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.194.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.194.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.194.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.62.keytab.
Entry for principal kudu/127.29.194.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.62.keytab.
I20260807 15:14:06.092583 30472 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7fZSIL/build/release/bin/kudu
/tmp/dist-test-task7fZSIL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115480851507-30472-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115480851507-30472-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115480851507-30472-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115480851507-30472-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.194.62:33665
--webserver_interface=127.29.194.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.62.keytab
--principal=kudu/127.29.194.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.194.20:41847
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.194.62:33665,127.29.194.61:38713,127.29.194.60:42169
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115480851507-30472-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115480851507-30472-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115480851507-30472-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:14:06.178370 17801 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:14:06.178594 17801 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:14:06.178618 17801 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:14:06.179873 17801 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:14:06.179919 17801 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:14:06.179934 17801 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:14:06.179945 17801 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:14:06.179956 17801 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:14:06.181650 17801 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.194.20:41847
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115480851507-30472-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115480851507-30472-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.194.62:33665,127.29.194.61:38713,127.29.194.60:42169
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.62.keytab
--principal=kudu/127.29.194.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.194.62:33665
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115480851507-30472-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.194.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17801
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115480851507-30472-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:14:06.181921 17801 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:14:06.182163 17801 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:14:06.187634 17813 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:14:06.187598 17815 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:14:06.187614 17817 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:14:06.187676 17801 server_base.cc:1034] running on GCE node
I20260807 15:14:06.188212 17801 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:14:06.188625 17801 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:14:06.189807 17801 hybrid_clock.cc:648] HybridClock initialized: now 1786115646189786 us; error 37 us; skew 500 ppm
Aug 07 15:14:06 dist-test-slave-42z9 krb5kdc[17779](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115646, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.194.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:14:06.196710 17801 init.cc:377] Logged in from keytab as kudu/127.29.194.62@KRBTEST.COM (short username kudu)
I20260807 15:14:06.197209 17801 webserver.cc:460] Webserver started at http://127.29.194.62:45441/ using document root <none> and password file <none>
I20260807 15:14:06.197499 17801 fs_manager.cc:359] Metadata directory not provided
I20260807 15:14:06.197552 17801 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:14:06.197692 17801 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:14:06.198837 17801 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115480851507-30472-0/minicluster-data/master-0/data/instance:
uuid: "1f1ebc2f7a7a40a598a4c98d85e350ab"
format_stamp: "Formatted at 2026-08-07 15:14:06 on dist-test-slave-42z9"
server_key: "ca3edd69e7e1e6e3167ef586936f0b15"
server_key_iv: "b2ac37be3db08f7a430bd92582dbd98a"
server_key_version: "encryptionkey@0"
I20260807 15:14:06.199204 17801 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115480851507-30472-0/minicluster-data/master-0/wal/instance:
uuid: "1f1ebc2f7a7a40a598a4c98d85e350ab"
format_stamp: "Formatted at 2026-08-07 15:14:06 on dist-test-slave-42z9"
server_key: "ca3edd69e7e1e6e3167ef586936f0b15"
server_key_iv: "b2ac37be3db08f7a430bd92582dbd98a"
server_key_version: "encryptionkey@0"
I20260807 15:14:06.200712 17801 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 15:14:06.201757 17830 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:14:06.201943 17801 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:14:06.202045 17801 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115480851507-30472-0/minicluster-data/master-0/data,/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115480851507-30472-0/minicluster-data/master-0/wal
uuid: "1f1ebc2f7a7a40a598a4c98d85e350ab"
format_stamp: "Formatted at 2026-08-07 15:14:06 on dist-test-slave-42z9"
server_key: "ca3edd69e7e1e6e3167ef586936f0b15"
server_key_iv: "b2ac37be3db08f7a430bd92582dbd98a"
server_key_version: "encryptionkey@0"
I20260807 15:14:06.202109 17801 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115480851507-30472-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115480851507-30472-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115480851507-30472-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:14:06.221011 17801 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:14:06.223305 17801 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:14:06.223435 17801 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:14:06.227506 17801 rpc_server.cc:307] RPC server started. Bound to: 127.29.194.62:33665
I20260807 15:14:06.227602 17923 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.194.62:33665 every 8 connection(s)
I20260807 15:14:06.227962 17801 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115480851507-30472-0/minicluster-data/master-0/data/info.pb
I20260807 15:14:06.228994 17924 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:14:06.229060 30472 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7fZSIL/build/release/bin/kudu as pid 17801
I20260807 15:14:06.229179 30472 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115480851507-30472-0/minicluster-data/master-0/wal/instance
I20260807 15:14:06.229363 30472 external_mini_cluster.cc:1428] Setting key e014f743cdcbccc93c54dfacb945213f
I20260807 15:14:06.230695 17924 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.194.62" port: 33665 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 15:14:06 dist-test-slave-42z9 krb5kdc[17779](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115646, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.194.62@KRBTEST.COM for kudu/127.29.194.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.29.194.61@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.194.61@KRBTEST.COM" created.
I20260807 15:14:06.241149 17927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:06.231484 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:33665 (local address 127.0.0.1:40216)
0807 15:14:06.231693 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0807 15:14:06.231731 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0807 15:14:06.232206 (+ 475us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:14:06.232494 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:14:06.232497 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:14:06.232781 (+ 284us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:14:06.232943 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:14:06.232947 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:14:06.233987 (+ 1040us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:14:06.233988 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:14:06.234670 (+ 682us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:14:06.234673 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:14:06.235133 (+ 460us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:14:06.235134 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:14:06.235196 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:06.235687 (+ 491us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:14:06.235695 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:14:06.237187 (+ 1492us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:14:06.239046 (+ 1859us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:14:06.239052 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:14:06.239054 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:14:06.239261 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:14:06.239550 (+ 289us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:14:06.239551 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:14:06.239551 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:14:06.239625 (+ 74us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:14:06.240190 (+ 565us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:14:06.240207 (+ 17us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:14:06.240354 (+ 147us) client_negotiation.cc:769] Sending connection context
0807 15:14:06.240531 (+ 177us) client_negotiation.cc:240] Negotiation successful
0807 15:14:06.240896 (+ 365us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":97,"threads_started":1}
I20260807 15:14:06.241153 17930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:06.231543 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40216 (local address 127.29.194.62:33665)
0807 15:14:06.231684 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0807 15:14:06.231695 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 15:14:06.231820 (+ 125us) server_negotiation.cc:407] Connection header received
0807 15:14:06.232334 (+ 514us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:14:06.232336 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:14:06.232353 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:14:06.232463 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:14:06.233043 (+ 580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:06.233895 (+ 852us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:06.234749 (+ 854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:06.235061 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:06.235156 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:06.237317 (+ 2161us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:14:06.237326 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:14:06.237326 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:14:06.237334 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:14:06.238955 (+ 1621us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:14:06.239397 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:14:06.239400 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:14:06.239400 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:14:06.239489 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:14:06.239675 (+ 186us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:14:06.239675 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:14:06.239676 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:14:06.239994 (+ 318us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:14:06.240074 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0807 15:14:06.240645 (+ 571us) server_negotiation.cc:299] Negotiation successful
0807 15:14:06.241042 (+ 397us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":238,"server-negotiator.queue_time_us":100,"thread_start_us":56,"threads_started":1}
I20260807 15:14:06.241990 17924 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.194.61" port: 38713 } has no permanent_uuid. Determining permanent_uuid...
I20260807 15:14:06.242379 17927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:06.242251 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.61:38713 (local address 127.0.0.1:59028)
0807 15:14:06.242294 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0807 15:14:06.242338 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.194.61:38713: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
W20260807 15:14:06.242491 17838 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.29.194.61:38713: connect: Connection refused (error 111)
I20260807 15:14:06.242904 17927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:06.242808 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.61:38713 (local address 127.0.0.1:59044)
0807 15:14:06.242845 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0807 15:14:06.242879 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.194.61:38713: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
W20260807 15:14:06.243060 17924 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.194.61:38713: Network error: Client connection negotiation failed: client connection to 127.29.194.61:38713: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.194.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.61.keytab.
Entry for principal kudu/127.29.194.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.61.keytab.
I20260807 15:14:06.256672 30472 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7fZSIL/build/release/bin/kudu
/tmp/dist-test-task7fZSIL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115480851507-30472-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115480851507-30472-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115480851507-30472-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115480851507-30472-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.194.61:38713
--webserver_interface=127.29.194.61
--webserver_port=0
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.61.keytab
--principal=kudu/127.29.194.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.194.20:41847
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.194.62:33665,127.29.194.61:38713,127.29.194.60:42169
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115480851507-30472-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115480851507-30472-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115480851507-30472-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 15:14:06.292318 17924 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.194.61" port: 38713 } attempt: 1
I20260807 15:14:06.292896 17927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:06.292748 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.61:38713 (local address 127.0.0.1:59060)
0807 15:14:06.292819 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0807 15:14:06.292867 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.194.61:38713: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260807 15:14:06.293252 17927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:06.293131 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.61:38713 (local address 127.0.0.1:59076)
0807 15:14:06.293194 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0807 15:14:06.293226 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.194.61:38713: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260807 15:14:06.293488 17924 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.194.61:38713: Network error: Client connection negotiation failed: client connection to 127.29.194.61:38713: connect: Connection refused (error 111)
W20260807 15:14:06.343580 17933 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:14:06.343806 17933 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:14:06.343830 17933 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:14:06.345027 17933 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:14:06.345080 17933 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:14:06.345093 17933 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:14:06.345104 17933 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:14:06.345115 17933 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:14:06.346751 17933 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.194.20:41847
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115480851507-30472-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115480851507-30472-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.29.194.62:33665,127.29.194.61:38713,127.29.194.60:42169
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.61.keytab
--principal=kudu/127.29.194.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.194.61:38713
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115480851507-30472-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.29.194.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17933
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115480851507-30472-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:14:06.347015 17933 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:14:06.347239 17933 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:14:06.352583 17942 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:14:06.352593 17938 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:14:06.352593 17944 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:14:06.353063 17933 server_base.cc:1034] running on GCE node
I20260807 15:14:06.353341 17933 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:14:06.353700 17933 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:14:06.354868 17933 hybrid_clock.cc:648] HybridClock initialized: now 1786115646354853 us; error 37 us; skew 500 ppm
Aug 07 15:14:06 dist-test-slave-42z9 krb5kdc[17779](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115646, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.194.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:14:06.357808 17933 init.cc:377] Logged in from keytab as kudu/127.29.194.61@KRBTEST.COM (short username kudu)
I20260807 15:14:06.358520 17933 webserver.cc:460] Webserver started at http://127.29.194.61:39913/ using document root <none> and password file <none>
I20260807 15:14:06.358852 17933 fs_manager.cc:359] Metadata directory not provided
I20260807 15:14:06.358916 17933 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:14:06.359086 17933 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:14:06.360451 17933 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115480851507-30472-0/minicluster-data/master-1/data/instance:
uuid: "b978bb5878a244428c9c03f7d108395a"
format_stamp: "Formatted at 2026-08-07 15:14:06 on dist-test-slave-42z9"
server_key: "42e75d83f2baf4d42bf6111842f5545b"
server_key_iv: "4b23ee92b7dbd9cb3f40ce0301bc51ab"
server_key_version: "encryptionkey@0"
I20260807 15:14:06.360891 17933 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115480851507-30472-0/minicluster-data/master-1/wal/instance:
uuid: "b978bb5878a244428c9c03f7d108395a"
format_stamp: "Formatted at 2026-08-07 15:14:06 on dist-test-slave-42z9"
server_key: "42e75d83f2baf4d42bf6111842f5545b"
server_key_iv: "4b23ee92b7dbd9cb3f40ce0301bc51ab"
server_key_version: "encryptionkey@0"
I20260807 15:14:06.361627 17924 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.194.61" port: 38713 } attempt: 2
I20260807 15:14:06.362015 17927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:06.361892 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.61:38713 (local address 127.0.0.1:59084)
0807 15:14:06.361948 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0807 15:14:06.361990 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.194.61:38713: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260807 15:14:06.362463 17927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:06.362306 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.61:38713 (local address 127.0.0.1:59098)
0807 15:14:06.362405 (+ 99us) negotiation.cc:107] Waiting for socket to connect
0807 15:14:06.362438 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.194.61:38713: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79}
W20260807 15:14:06.362560 17924 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.194.61:38713: Network error: Client connection negotiation failed: client connection to 127.29.194.61:38713: connect: Connection refused (error 111)
I20260807 15:14:06.362653 17933 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.004s sys 0.000s
I20260807 15:14:06.363433 17954 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:14:06.363649 17933 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:14:06.363745 17933 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115480851507-30472-0/minicluster-data/master-1/data,/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115480851507-30472-0/minicluster-data/master-1/wal
uuid: "b978bb5878a244428c9c03f7d108395a"
format_stamp: "Formatted at 2026-08-07 15:14:06 on dist-test-slave-42z9"
server_key: "42e75d83f2baf4d42bf6111842f5545b"
server_key_iv: "4b23ee92b7dbd9cb3f40ce0301bc51ab"
server_key_version: "encryptionkey@0"
I20260807 15:14:06.363819 17933 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115480851507-30472-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115480851507-30472-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115480851507-30472-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:14:06.378943 17933 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:14:06.380986 17933 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:14:06.381129 17933 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:14:06.385134 17933 rpc_server.cc:307] RPC server started. Bound to: 127.29.194.61:38713
I20260807 15:14:06.385263 18036 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.194.61:38713 every 8 connection(s)
I20260807 15:14:06.385576 17933 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115480851507-30472-0/minicluster-data/master-1/data/info.pb
I20260807 15:14:06.386523 18037 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:14:06.388084 18037 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.194.62" port: 33665 } has no permanent_uuid. Determining permanent_uuid...
I20260807 15:14:06.393039 30472 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7fZSIL/build/release/bin/kudu as pid 17933
I20260807 15:14:06.393157 30472 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115480851507-30472-0/minicluster-data/master-1/wal/instance
I20260807 15:14:06.393328 30472 external_mini_cluster.cc:1428] Setting key 68cd77a9d890defe01dc3b3268df7e71
Aug 07 15:14:06 dist-test-slave-42z9 krb5kdc[17779](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115646, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.194.61@KRBTEST.COM for kudu/127.29.194.62@KRBTEST.COM
I20260807 15:14:06.397835 17930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:06.388864 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40222 (local address 127.29.194.62:33665)
0807 15:14:06.388931 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0807 15:14:06.388933 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:14:06.389113 (+ 180us) server_negotiation.cc:407] Connection header received
0807 15:14:06.389658 (+ 545us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:14:06.389659 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:14:06.389672 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:14:06.389731 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:14:06.390355 (+ 624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:06.390857 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:06.391863 (+ 1006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:06.392085 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:06.392149 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:06.394402 (+ 2253us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:14:06.394404 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:14:06.394404 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:14:06.394412 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:14:06.396092 (+ 1680us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:14:06.396643 (+ 551us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:14:06.396644 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:14:06.396645 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:14:06.396717 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:14:06.397022 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:14:06.397023 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:14:06.397023 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:14:06.397159 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:14:06.397215 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0807 15:14:06.397639 (+ 424us) server_negotiation.cc:299] Negotiation successful
0807 15:14:06.397738 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260807 15:14:06.398029 18039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:06.388815 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:33665 (local address 127.0.0.1:40222)
0807 15:14:06.388997 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0807 15:14:06.389034 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0807 15:14:06.389543 (+ 509us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:14:06.389794 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:14:06.389797 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:14:06.390063 (+ 266us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:14:06.390261 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:14:06.390265 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:14:06.390966 (+ 701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:14:06.390967 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:14:06.391770 (+ 803us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:14:06.391773 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:14:06.392158 (+ 385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:14:06.392159 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:14:06.392261 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:06.392778 (+ 517us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:14:06.392786 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:14:06.394309 (+ 1523us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:14:06.396223 (+ 1914us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:14:06.396230 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:14:06.396232 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:14:06.396532 (+ 300us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:14:06.396793 (+ 261us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:14:06.396794 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:14:06.396794 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:14:06.396932 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:14:06.397220 (+ 288us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:14:06.397227 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:14:06.397468 (+ 241us) client_negotiation.cc:769] Sending connection context
0807 15:14:06.397602 (+ 134us) client_negotiation.cc:240] Negotiation successful
0807 15:14:06.397786 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":90,"threads_started":1}
I20260807 15:14:06.398844 18037 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.194.61" port: 38713 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.29.194.60@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.194.60@KRBTEST.COM" created.
Aug 07 15:14:06 dist-test-slave-42z9 krb5kdc[17779](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115646, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.194.61@KRBTEST.COM for kudu/127.29.194.61@KRBTEST.COM
I20260807 15:14:06.407966 18039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:06.399146 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.61:38713 (local address 127.0.0.1:59114)
0807 15:14:06.399312 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0807 15:14:06.399326 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 15:14:06.399547 (+ 221us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:14:06.399847 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:14:06.399848 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:14:06.400060 (+ 212us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:14:06.400140 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:14:06.400142 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:14:06.401171 (+ 1029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:14:06.401173 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:14:06.401717 (+ 544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:14:06.401719 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:14:06.402156 (+ 437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:14:06.402157 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:14:06.402225 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:06.402620 (+ 395us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:14:06.402625 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:14:06.405211 (+ 2586us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:14:06.407001 (+ 1790us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:14:06.407002 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:14:06.407002 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:14:06.407151 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:14:06.407327 (+ 176us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:14:06.407328 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:14:06.407328 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:14:06.407378 (+ 50us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:14:06.407638 (+ 260us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:14:06.407638 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:14:06.407677 (+ 39us) client_negotiation.cc:769] Sending connection context
0807 15:14:06.407740 (+ 63us) client_negotiation.cc:240] Negotiation successful
0807 15:14:06.407853 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":78}
I20260807 15:14:06.408150 18046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:06.399391 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59114 (local address 127.29.194.61:38713)
0807 15:14:06.399658 (+ 267us) server_negotiation.cc:206] Beginning negotiation
0807 15:14:06.399665 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 15:14:06.399710 (+ 45us) server_negotiation.cc:407] Connection header received
0807 15:14:06.399734 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:14:06.399737 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:14:06.399751 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:14:06.399833 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:14:06.400314 (+ 481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:06.401079 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:06.401801 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:06.402090 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:06.402139 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:06.405325 (+ 3186us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:14:06.405336 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:14:06.405337 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:14:06.405344 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:14:06.406898 (+ 1554us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:14:06.407221 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:14:06.407224 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:14:06.407224 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:14:06.407274 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:14:06.407426 (+ 152us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:14:06.407426 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:14:06.407427 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:14:06.407579 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:14:06.407650 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0807 15:14:06.407775 (+ 125us) server_negotiation.cc:299] Negotiation successful
0807 15:14:06.407927 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":63,"server-negotiator.queue_time_us":215,"thread_start_us":96,"threads_started":1}
I20260807 15:14:06.408712 18037 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.194.60" port: 42169 } has no permanent_uuid. Determining permanent_uuid...
I20260807 15:14:06.409031 18039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:06.408926 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.60:42169 (local address 127.0.0.1:59708)
0807 15:14:06.408969 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0807 15:14:06.409012 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.194.60:42169: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260807 15:14:06.409142 17955 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.29.194.60:42169: connect: Connection refused (error 111)
I20260807 15:14:06.409543 18039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:06.409420 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.60:42169 (local address 127.0.0.1:59720)
0807 15:14:06.409483 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0807 15:14:06.409518 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.194.60:42169: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260807 15:14:06.409651 18037 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.194.60:42169: Network error: Client connection negotiation failed: client connection to 127.29.194.60:42169: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.194.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.60.keytab.
Entry for principal kudu/127.29.194.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.60.keytab.
I20260807 15:14:06.421104 30472 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7fZSIL/build/release/bin/kudu
/tmp/dist-test-task7fZSIL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115480851507-30472-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115480851507-30472-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115480851507-30472-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115480851507-30472-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.194.60:42169
--webserver_interface=127.29.194.60
--webserver_port=0
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.60.keytab
--principal=kudu/127.29.194.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.194.20:41847
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.194.62:33665,127.29.194.61:38713,127.29.194.60:42169
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115480851507-30472-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115480851507-30472-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115480851507-30472-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 15:14:06.453740 17924 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.194.61" port: 38713 } attempt: 3
Aug 07 15:14:06 dist-test-slave-42z9 krb5kdc[17779](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115646, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.194.62@KRBTEST.COM for kudu/127.29.194.61@KRBTEST.COM
I20260807 15:14:06.458941 18037 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.194.60" port: 42169 } attempt: 1
I20260807 15:14:06.459474 18039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:06.459317 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.60:42169 (local address 127.0.0.1:59736)
0807 15:14:06.459401 (+ 84us) negotiation.cc:107] Waiting for socket to connect
0807 15:14:06.459439 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.194.60:42169: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260807 15:14:06.459828 18039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:06.459734 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.60:42169 (local address 127.0.0.1:59744)
0807 15:14:06.459775 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0807 15:14:06.459802 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.194.60:42169: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260807 15:14:06.460234 18037 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.194.60:42169: Network error: Client connection negotiation failed: client connection to 127.29.194.60:42169: connect: Connection refused (error 111)
I20260807 15:14:06.460964 18046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:06.454283 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59126 (local address 127.29.194.61:38713)
0807 15:14:06.454372 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0807 15:14:06.454375 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:14:06.454383 (+ 8us) server_negotiation.cc:407] Connection header received
0807 15:14:06.454408 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:14:06.454409 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:14:06.454429 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:14:06.454484 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:14:06.454811 (+ 327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:06.455332 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:06.456000 (+ 668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:06.456257 (+ 257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:06.456321 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:06.458217 (+ 1896us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:14:06.458219 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:14:06.458219 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:14:06.458228 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:14:06.459757 (+ 1529us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:14:06.460206 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:14:06.460207 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:14:06.460208 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:14:06.460294 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:14:06.460497 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:14:06.460498 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:14:06.460498 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:14:06.460626 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:14:06.460679 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0807 15:14:06.460835 (+ 156us) server_negotiation.cc:299] Negotiation successful
0807 15:14:06.460908 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260807 15:14:06.461031 17927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:06.454161 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.61:38713 (local address 127.0.0.1:59126)
0807 15:14:06.454216 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0807 15:14:06.454228 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 15:14:06.454282 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:14:06.454494 (+ 212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:14:06.454495 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:14:06.454668 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:14:06.454736 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:14:06.454738 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:14:06.455408 (+ 670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:14:06.455410 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:14:06.455921 (+ 511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:14:06.455922 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:14:06.456322 (+ 400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:14:06.456322 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:14:06.456396 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:06.456834 (+ 438us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:14:06.456839 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:14:06.458129 (+ 1290us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:14:06.459871 (+ 1742us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:14:06.459872 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:14:06.459873 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:14:06.460109 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:14:06.460365 (+ 256us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:14:06.460366 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:14:06.460366 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:14:06.460437 (+ 71us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:14:06.460686 (+ 249us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:14:06.460687 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:14:06.460744 (+ 57us) client_negotiation.cc:769] Sending connection context
0807 15:14:06.460818 (+ 74us) client_negotiation.cc:240] Negotiation successful
0807 15:14:06.460934 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31}
I20260807 15:14:06.461561 17924 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.194.60" port: 42169 } has no permanent_uuid. Determining permanent_uuid...
I20260807 15:14:06.461925 17927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:06.461811 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.60:42169 (local address 127.0.0.1:59758)
0807 15:14:06.461861 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0807 15:14:06.461897 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.194.60:42169: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
I20260807 15:14:06.462271 17927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:06.462180 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.60:42169 (local address 127.0.0.1:59772)
0807 15:14:06.462223 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0807 15:14:06.462251 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.194.60:42169: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260807 15:14:06.462494 17924 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.194.60:42169: Network error: Client connection negotiation failed: client connection to 127.29.194.60:42169: connect: Connection refused (error 111)
I20260807 15:14:06.493752 17924 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.194.60" port: 42169 } attempt: 1
I20260807 15:14:06.494341 17927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:06.494165 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.60:42169 (local address 127.0.0.1:59774)
0807 15:14:06.494259 (+ 94us) negotiation.cc:107] Waiting for socket to connect
0807 15:14:06.494309 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.194.60:42169: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260807 15:14:06.494721 17927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:06.494627 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.60:42169 (local address 127.0.0.1:59776)
0807 15:14:06.494671 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0807 15:14:06.494704 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.194.60:42169: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260807 15:14:06.494815 17924 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.194.60:42169: Network error: Client connection negotiation failed: client connection to 127.29.194.60:42169: connect: Connection refused (error 111)
W20260807 15:14:06.510865 18050 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:14:06.511097 18050 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:14:06.511121 18050 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:14:06.512259 18050 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:14:06.512300 18050 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:14:06.512313 18050 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:14:06.512326 18050 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:14:06.512336 18050 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:14:06.514005 18050 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.194.20:41847
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115480851507-30472-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115480851507-30472-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.29.194.62:33665,127.29.194.61:38713,127.29.194.60:42169
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.60.keytab
--principal=kudu/127.29.194.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.194.60:42169
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115480851507-30472-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.29.194.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18050
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115480851507-30472-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:14:06.514285 18050 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:14:06.514526 18050 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:14:06.519446 18058 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:14:06.519577 18050 server_base.cc:1034] running on GCE node
W20260807 15:14:06.519465 18060 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:14:06.519713 18057 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:14:06.519964 18050 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:14:06.520249 18050 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:14:06.521416 18050 hybrid_clock.cc:648] HybridClock initialized: now 1786115646521398 us; error 41 us; skew 500 ppm
Aug 07 15:14:06 dist-test-slave-42z9 krb5kdc[17779](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115646, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.194.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:14:06.524191 18050 init.cc:377] Logged in from keytab as kudu/127.29.194.60@KRBTEST.COM (short username kudu)
I20260807 15:14:06.524636 18050 webserver.cc:460] Webserver started at http://127.29.194.60:46535/ using document root <none> and password file <none>
I20260807 15:14:06.524844 18050 fs_manager.cc:359] Metadata directory not provided
I20260807 15:14:06.524883 18050 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:14:06.524982 18050 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:14:06.526090 18050 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115480851507-30472-0/minicluster-data/master-2/data/instance:
uuid: "28c75a9c172140c4b412aaf9629fabf1"
format_stamp: "Formatted at 2026-08-07 15:14:06 on dist-test-slave-42z9"
server_key: "e573c6cb541b59872c8ad0f80405d417"
server_key_iv: "7bd28b17d258e431b7bff8b23c871cd6"
server_key_version: "encryptionkey@0"
I20260807 15:14:06.526392 18050 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115480851507-30472-0/minicluster-data/master-2/wal/instance:
uuid: "28c75a9c172140c4b412aaf9629fabf1"
format_stamp: "Formatted at 2026-08-07 15:14:06 on dist-test-slave-42z9"
server_key: "e573c6cb541b59872c8ad0f80405d417"
server_key_iv: "7bd28b17d258e431b7bff8b23c871cd6"
server_key_version: "encryptionkey@0"
I20260807 15:14:06.527745 18050 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:14:06.528429 18037 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.194.60" port: 42169 } attempt: 2
I20260807 15:14:06.528484 18072 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:14:06.528843 18039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:06.528723 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.60:42169 (local address 127.0.0.1:59790)
0807 15:14:06.528771 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0807 15:14:06.528812 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.194.60:42169: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
I20260807 15:14:06.528829 18050 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:14:06.528929 18050 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115480851507-30472-0/minicluster-data/master-2/data,/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115480851507-30472-0/minicluster-data/master-2/wal
uuid: "28c75a9c172140c4b412aaf9629fabf1"
format_stamp: "Formatted at 2026-08-07 15:14:06 on dist-test-slave-42z9"
server_key: "e573c6cb541b59872c8ad0f80405d417"
server_key_iv: "7bd28b17d258e431b7bff8b23c871cd6"
server_key_version: "encryptionkey@0"
I20260807 15:14:06.529026 18050 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115480851507-30472-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115480851507-30472-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115480851507-30472-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:14:06.529228 18039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:06.529136 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.60:42169 (local address 127.0.0.1:59796)
0807 15:14:06.529180 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0807 15:14:06.529203 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.194.60:42169: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260807 15:14:06.529397 18037 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.194.60:42169: Network error: Client connection negotiation failed: client connection to 127.29.194.60:42169: connect: Connection refused (error 111)
I20260807 15:14:06.542941 18050 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:14:06.545157 18050 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:14:06.545284 18050 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:14:06.549727 18050 rpc_server.cc:307] RPC server started. Bound to: 127.29.194.60:42169
I20260807 15:14:06.549772 18154 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.194.60:42169 every 8 connection(s)
I20260807 15:14:06.550158 18050 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115480851507-30472-0/minicluster-data/master-2/data/info.pb
I20260807 15:14:06.551497 18155 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:14:06.553617 18155 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.194.62" port: 33665 } has no permanent_uuid. Determining permanent_uuid...
I20260807 15:14:06.558226 30472 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7fZSIL/build/release/bin/kudu as pid 18050
I20260807 15:14:06.558342 30472 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115480851507-30472-0/minicluster-data/master-2/wal/instance
I20260807 15:14:06.558558 30472 external_mini_cluster.cc:1428] Setting key cf59ece17e3173ad06a0fad22e2ffe3d
Aug 07 15:14:06 dist-test-slave-42z9 krb5kdc[17779](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115646, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.194.60@KRBTEST.COM for kudu/127.29.194.62@KRBTEST.COM
Aug 07 15:14:06 dist-test-slave-42z9 krb5kdc[17779](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115646, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.194.60@KRBTEST.COM
I20260807 15:14:06.564903 17930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:06.554745 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40232 (local address 127.29.194.62:33665)
0807 15:14:06.554815 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0807 15:14:06.554817 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:14:06.555217 (+ 400us) server_negotiation.cc:407] Connection header received
0807 15:14:06.555853 (+ 636us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:14:06.555856 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:14:06.555867 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:14:06.555939 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:14:06.556833 (+ 894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:06.557376 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:06.558660 (+ 1284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:06.558930 (+ 270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:06.559022 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:06.562141 (+ 3119us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:14:06.562143 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:14:06.562144 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:14:06.562155 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:14:06.563530 (+ 1375us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:14:06.563946 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:14:06.563947 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:14:06.563947 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:14:06.564009 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:14:06.564240 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:14:06.564241 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:14:06.564241 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:14:06.564385 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:14:06.564451 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0807 15:14:06.564747 (+ 296us) server_negotiation.cc:299] Negotiation successful
0807 15:14:06.564832 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260807 15:14:06.565157 18157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:06.554698 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:33665 (local address 127.0.0.1:40232)
0807 15:14:06.555001 (+ 303us) negotiation.cc:107] Waiting for socket to connect
0807 15:14:06.555058 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0807 15:14:06.555720 (+ 662us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:14:06.556030 (+ 310us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:14:06.556034 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:14:06.556412 (+ 378us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:14:06.556693 (+ 281us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:14:06.556698 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:14:06.557521 (+ 823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:14:06.557522 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:14:06.558532 (+ 1010us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:14:06.558536 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:14:06.559051 (+ 515us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:14:06.559052 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:14:06.559269 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:06.560267 (+ 998us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:14:06.560284 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:14:06.562029 (+ 1745us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:14:06.563614 (+ 1585us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:14:06.563623 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:14:06.563626 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:14:06.563876 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:14:06.564066 (+ 190us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:14:06.564067 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:14:06.564067 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:14:06.564182 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:14:06.564457 (+ 275us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:14:06.564464 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:14:06.564610 (+ 146us) client_negotiation.cc:769] Sending connection context
0807 15:14:06.564706 (+ 96us) client_negotiation.cc:240] Negotiation successful
0807 15:14:06.564906 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":238,"thread_start_us":134,"threads_started":1}
I20260807 15:14:06.565788 18155 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.194.61" port: 38713 } has no permanent_uuid. Determining permanent_uuid...
I20260807 15:14:06.567905 18159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:06.559123 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59808 (local address 127.29.194.60:42169)
0807 15:14:06.559366 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0807 15:14:06.559373 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 15:14:06.559397 (+ 24us) server_negotiation.cc:407] Connection header received
0807 15:14:06.559417 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:14:06.559419 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:14:06.559439 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:14:06.559510 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:14:06.560161 (+ 651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:06.561019 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:06.561970 (+ 951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:06.562405 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:06.562493 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:06.564755 (+ 2262us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:14:06.564762 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:14:06.564762 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:14:06.564769 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:14:06.566744 (+ 1975us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:14:06.567145 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:14:06.567148 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:14:06.567149 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:14:06.567216 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:14:06.567410 (+ 194us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:14:06.567411 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:14:06.567411 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:14:06.567562 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:14:06.567693 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0807 15:14:06.567766 (+ 73us) server_negotiation.cc:299] Negotiation successful
0807 15:14:06.567830 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":192,"thread_start_us":144,"threads_started":1}
Aug 07 15:14:06 dist-test-slave-42z9 krb5kdc[17779](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115646, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.194.60@KRBTEST.COM for kudu/127.29.194.61@KRBTEST.COM
I20260807 15:14:06.569972 17924 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.194.60" port: 42169 } attempt: 2
I20260807 15:14:06.573902 18046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:06.566122 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59128 (local address 127.29.194.61:38713)
0807 15:14:06.566168 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0807 15:14:06.566170 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:14:06.566180 (+ 10us) server_negotiation.cc:407] Connection header received
0807 15:14:06.566207 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:14:06.566208 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:14:06.566219 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:14:06.566289 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:14:06.566556 (+ 267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:06.567211 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:06.568073 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:06.568301 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:06.568334 (+ 33us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:06.570293 (+ 1959us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:14:06.570295 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:14:06.570295 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:14:06.570303 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:14:06.571959 (+ 1656us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:14:06.572578 (+ 619us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:14:06.572633 (+ 55us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:14:06.572634 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:14:06.572727 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:14:06.573008 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:14:06.573010 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:14:06.573011 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:14:06.573483 (+ 472us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:14:06.573567 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0807 15:14:06.573715 (+ 148us) server_negotiation.cc:299] Negotiation successful
0807 15:14:06.573800 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260807 15:14:06.573904 18157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:06.566065 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.61:38713 (local address 127.0.0.1:59128)
0807 15:14:06.566114 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0807 15:14:06.566124 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 15:14:06.566157 (+ 33us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:14:06.566291 (+ 134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:14:06.566291 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:14:06.566417 (+ 126us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:14:06.566481 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:14:06.566483 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:14:06.567270 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:14:06.567272 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:14:06.567983 (+ 711us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:14:06.567987 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:14:06.568394 (+ 407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:14:06.568395 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:14:06.568445 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:06.568827 (+ 382us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:14:06.568833 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:14:06.570220 (+ 1387us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:14:06.572124 (+ 1904us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:14:06.572126 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:14:06.572127 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:14:06.572488 (+ 361us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:14:06.572806 (+ 318us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:14:06.572807 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:14:06.572808 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:14:06.572888 (+ 80us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:14:06.573562 (+ 674us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:14:06.573563 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:14:06.573618 (+ 55us) client_negotiation.cc:769] Sending connection context
0807 15:14:06.573682 (+ 64us) client_negotiation.cc:240] Negotiation successful
0807 15:14:06.573800 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
Aug 07 15:14:06 dist-test-slave-42z9 krb5kdc[17779](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115646, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.194.62@KRBTEST.COM
Aug 07 15:14:06 dist-test-slave-42z9 krb5kdc[17779](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115646, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.194.62@KRBTEST.COM for kudu/127.29.194.60@KRBTEST.COM
I20260807 15:14:06.575385 18155 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.194.60" port: 42169 } has no permanent_uuid. Determining permanent_uuid...
I20260807 15:14:06.578469 17927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:06.570227 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.60:42169 (local address 127.0.0.1:59818)
0807 15:14:06.570275 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0807 15:14:06.570284 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0807 15:14:06.570318 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:14:06.570576 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:14:06.570577 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:14:06.570717 (+ 140us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:14:06.570761 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:14:06.570762 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:14:06.571576 (+ 814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:14:06.571577 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:14:06.572365 (+ 788us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:14:06.572369 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:14:06.572922 (+ 553us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:14:06.572923 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:14:06.572994 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:06.573484 (+ 490us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:14:06.573489 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:14:06.575179 (+ 1690us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:14:06.577360 (+ 2181us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:14:06.577362 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:14:06.577362 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:14:06.577628 (+ 266us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:14:06.577811 (+ 183us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:14:06.577812 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:14:06.577812 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:14:06.577877 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:14:06.578097 (+ 220us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:14:06.578098 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:14:06.578155 (+ 57us) client_negotiation.cc:769] Sending connection context
0807 15:14:06.578237 (+ 82us) client_negotiation.cc:240] Negotiation successful
0807 15:14:06.578360 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32}
I20260807 15:14:06.578670 17930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:06.568795 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40234 (local address 127.29.194.62:33665)
0807 15:14:06.568834 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0807 15:14:06.568836 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:14:06.568843 (+ 7us) server_negotiation.cc:407] Connection header received
0807 15:14:06.568866 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:14:06.568867 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:14:06.568876 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:14:06.568918 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:14:06.569506 (+ 588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:06.570032 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:06.570876 (+ 844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:06.571081 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:06.571324 (+ 243us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:06.574656 (+ 3332us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:14:06.574658 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:14:06.574658 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:14:06.574668 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:14:06.576777 (+ 2109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:14:06.577269 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:14:06.577270 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:14:06.577271 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:14:06.577345 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:14:06.577706 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:14:06.577708 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:14:06.577708 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:14:06.577898 (+ 190us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:14:06.578158 (+ 260us) server_negotiation.cc:1035] Waiting for connection context
0807 15:14:06.578431 (+ 273us) server_negotiation.cc:299] Negotiation successful
0807 15:14:06.578535 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260807 15:14:06.578859 18159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:06.570415 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59818 (local address 127.29.194.60:42169)
0807 15:14:06.570452 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0807 15:14:06.570454 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:14:06.570461 (+ 7us) server_negotiation.cc:407] Connection header received
0807 15:14:06.570479 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:14:06.570479 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:14:06.570489 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:14:06.570563 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:14:06.570844 (+ 281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:06.571461 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:06.572498 (+ 1037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:06.572822 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:06.572909 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:06.575292 (+ 2383us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:14:06.575294 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:14:06.575294 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:14:06.575303 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:14:06.577230 (+ 1927us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:14:06.577696 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:14:06.577697 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:14:06.577698 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:14:06.577758 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:14:06.577927 (+ 169us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:14:06.577928 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:14:06.577928 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:14:06.578044 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:14:06.578099 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0807 15:14:06.578680 (+ 581us) server_negotiation.cc:299] Negotiation successful
0807 15:14:06.578772 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260807 15:14:06.581336 17924 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1f1ebc2f7a7a40a598a4c98d85e350ab: Bootstrap starting.
I20260807 15:14:06.582079 17924 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1f1ebc2f7a7a40a598a4c98d85e350ab: Neither blocks nor log segments found. Creating new log.
Aug 07 15:14:06 dist-test-slave-42z9 krb5kdc[17779](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115646, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.194.60@KRBTEST.COM for kudu/127.29.194.60@KRBTEST.COM
I20260807 15:14:06.582388 17924 log.cc:824] T 00000000000000000000000000000000 P 1f1ebc2f7a7a40a598a4c98d85e350ab: Log is configured to *not* fsync() on all Append() calls
I20260807 15:14:06.583576 17924 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1f1ebc2f7a7a40a598a4c98d85e350ab: No bootstrap required, opened a new log
Aug 07 15:14:06 dist-test-slave-42z9 krb5kdc[17779](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115646, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.194.61@KRBTEST.COM
I20260807 15:14:06.585943 17924 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1f1ebc2f7a7a40a598a4c98d85e350ab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f1ebc2f7a7a40a598a4c98d85e350ab" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 33665 } } peers { permanent_uuid: "b978bb5878a244428c9c03f7d108395a" member_type: VOTER last_known_addr { host: "127.29.194.61" port: 38713 } } peers { permanent_uuid: "28c75a9c172140c4b412aaf9629fabf1" member_type: VOTER last_known_addr { host: "127.29.194.60" port: 42169 } }
I20260807 15:14:06.586113 17924 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1f1ebc2f7a7a40a598a4c98d85e350ab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:14:06.586155 17924 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1f1ebc2f7a7a40a598a4c98d85e350ab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f1ebc2f7a7a40a598a4c98d85e350ab, State: Initialized, Role: FOLLOWER
I20260807 15:14:06.586283 17924 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1f1ebc2f7a7a40a598a4c98d85e350ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f1ebc2f7a7a40a598a4c98d85e350ab" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 33665 } } peers { permanent_uuid: "b978bb5878a244428c9c03f7d108395a" member_type: VOTER last_known_addr { host: "127.29.194.61" port: 38713 } } peers { permanent_uuid: "28c75a9c172140c4b412aaf9629fabf1" member_type: VOTER last_known_addr { host: "127.29.194.60" port: 42169 } }
I20260807 15:14:06.586371 18157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:06.576311 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.60:42169 (local address 127.0.0.1:59834)
0807 15:14:06.576389 (+ 78us) negotiation.cc:107] Waiting for socket to connect
0807 15:14:06.576401 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 15:14:06.576452 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:14:06.576901 (+ 449us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:14:06.576903 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:14:06.577166 (+ 263us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:14:06.577225 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:14:06.577238 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:14:06.578993 (+ 1755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:14:06.578994 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:14:06.579698 (+ 704us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:14:06.579701 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:14:06.580363 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:14:06.580364 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:14:06.580414 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:06.580853 (+ 439us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:14:06.580858 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:14:06.582501 (+ 1643us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:14:06.585122 (+ 2621us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:14:06.585123 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:14:06.585124 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:14:06.585348 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:14:06.585631 (+ 283us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:14:06.585633 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:14:06.585633 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:14:06.585709 (+ 76us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:14:06.586078 (+ 369us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:14:06.586079 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:14:06.586122 (+ 43us) client_negotiation.cc:769] Sending connection context
0807 15:14:06.586169 (+ 47us) client_negotiation.cc:240] Negotiation successful
0807 15:14:06.586279 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":54}
I20260807 15:14:06.586649 17924 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1f1ebc2f7a7a40a598a4c98d85e350ab [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:14:06.586845 18161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:06.576505 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59834 (local address 127.29.194.60:42169)
0807 15:14:06.576740 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0807 15:14:06.576743 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:14:06.576752 (+ 9us) server_negotiation.cc:407] Connection header received
0807 15:14:06.576773 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:14:06.576774 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:14:06.576787 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:14:06.576868 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:14:06.577339 (+ 471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:06.578883 (+ 1544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:06.580012 (+ 1129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:06.580274 (+ 262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:06.580603 (+ 329us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:06.582598 (+ 1995us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:14:06.582599 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:14:06.582600 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:14:06.582607 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:14:06.585004 (+ 2397us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:14:06.585425 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:14:06.585426 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:14:06.585426 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:14:06.585553 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:14:06.585783 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:14:06.585784 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:14:06.585784 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:14:06.585995 (+ 211us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:14:06.586596 (+ 601us) server_negotiation.cc:1035] Waiting for connection context
0807 15:14:06.586638 (+ 42us) server_negotiation.cc:299] Negotiation successful
0807 15:14:06.586721 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":207,"thread_start_us":139,"threads_started":1}
I20260807 15:14:06.586802 18164 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1f1ebc2f7a7a40a598a4c98d85e350ab [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f1ebc2f7a7a40a598a4c98d85e350ab" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 33665 } } peers { permanent_uuid: "b978bb5878a244428c9c03f7d108395a" member_type: VOTER last_known_addr { host: "127.29.194.61" port: 38713 } } peers { permanent_uuid: "28c75a9c172140c4b412aaf9629fabf1" member_type: VOTER last_known_addr { host: "127.29.194.60" port: 42169 } } }
I20260807 15:14:06.586889 18164 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1f1ebc2f7a7a40a598a4c98d85e350ab [sys.catalog]: This master's current role is: FOLLOWER
I20260807 15:14:06.588426 18155 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 28c75a9c172140c4b412aaf9629fabf1: Bootstrap starting.
W20260807 15:14:06.588459 18182 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 1f1ebc2f7a7a40a598a4c98d85e350ab: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 15:14:06.588507 18182 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 15:14:06.588863 18046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:06.580426 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59134 (local address 127.29.194.61:38713)
0807 15:14:06.580539 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0807 15:14:06.580542 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:14:06.580555 (+ 13us) server_negotiation.cc:407] Connection header received
0807 15:14:06.580682 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:14:06.580683 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:14:06.580697 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:14:06.580755 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:14:06.581519 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:06.582251 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:06.583217 (+ 966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:06.583484 (+ 267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:06.583575 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:06.585908 (+ 2333us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:14:06.585910 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:14:06.585911 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:14:06.585920 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:14:06.587328 (+ 1408us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:14:06.587712 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:14:06.587714 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:14:06.587714 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:14:06.587771 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:14:06.587954 (+ 183us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:14:06.587955 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:14:06.587956 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:14:06.588093 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:14:06.588273 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0807 15:14:06.588684 (+ 411us) server_negotiation.cc:299] Negotiation successful
0807 15:14:06.588772 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260807 15:14:06.589214 18155 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 28c75a9c172140c4b412aaf9629fabf1: Neither blocks nor log segments found. Creating new log.
I20260807 15:14:06.589488 18155 log.cc:824] T 00000000000000000000000000000000 P 28c75a9c172140c4b412aaf9629fabf1: Log is configured to *not* fsync() on all Append() calls
I20260807 15:14:06.590327 30472 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 15:14:06.590629 18155 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 28c75a9c172140c4b412aaf9629fabf1: No bootstrap required, opened a new log
I20260807 15:14:06.592334 18155 raft_consensus.cc:348] T 00000000000000000000000000000000 P 28c75a9c172140c4b412aaf9629fabf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f1ebc2f7a7a40a598a4c98d85e350ab" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 33665 } } peers { permanent_uuid: "b978bb5878a244428c9c03f7d108395a" member_type: VOTER last_known_addr { host: "127.29.194.61" port: 38713 } } peers { permanent_uuid: "28c75a9c172140c4b412aaf9629fabf1" member_type: VOTER last_known_addr { host: "127.29.194.60" port: 42169 } }
I20260807 15:14:06.593135 17801 master_runner.cc:186] Error getting master registration for 127.29.194.61:38713: OK, instance_id { permanent_uuid: "b978bb5878a244428c9c03f7d108395a" instance_seqno: 1786115646384010 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260807 15:14:06.595690 17801 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260807 15:14:06.595867 18155 raft_consensus.cc:374] T 00000000000000000000000000000000 P 28c75a9c172140c4b412aaf9629fabf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:14:06.596406 18155 raft_consensus.cc:729] T 00000000000000000000000000000000 P 28c75a9c172140c4b412aaf9629fabf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 28c75a9c172140c4b412aaf9629fabf1, State: Initialized, Role: FOLLOWER
I20260807 15:14:06.596704 18155 consensus_queue.cc:260] T 00000000000000000000000000000000 P 28c75a9c172140c4b412aaf9629fabf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f1ebc2f7a7a40a598a4c98d85e350ab" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 33665 } } peers { permanent_uuid: "b978bb5878a244428c9c03f7d108395a" member_type: VOTER last_known_addr { host: "127.29.194.61" port: 38713 } } peers { permanent_uuid: "28c75a9c172140c4b412aaf9629fabf1" member_type: VOTER last_known_addr { host: "127.29.194.60" port: 42169 } }
I20260807 15:14:06.597219 18155 sys_catalog.cc:564] T 00000000000000000000000000000000 P 28c75a9c172140c4b412aaf9629fabf1 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:14:06.597911 18193 sys_catalog.cc:455] T 00000000000000000000000000000000 P 28c75a9c172140c4b412aaf9629fabf1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f1ebc2f7a7a40a598a4c98d85e350ab" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 33665 } } peers { permanent_uuid: "b978bb5878a244428c9c03f7d108395a" member_type: VOTER last_known_addr { host: "127.29.194.61" port: 38713 } } peers { permanent_uuid: "28c75a9c172140c4b412aaf9629fabf1" member_type: VOTER last_known_addr { host: "127.29.194.60" port: 42169 } } }
I20260807 15:14:06.598445 18193 sys_catalog.cc:458] T 00000000000000000000000000000000 P 28c75a9c172140c4b412aaf9629fabf1 [sys.catalog]: This master's current role is: FOLLOWER
W20260807 15:14:06.600485 18207 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 28c75a9c172140c4b412aaf9629fabf1: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 15:14:06.600546 18207 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 15:14:06.601078 18161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:06.592024 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59842 (local address 127.29.194.60:42169)
0807 15:14:06.592063 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0807 15:14:06.592064 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 15:14:06.592557 (+ 493us) server_negotiation.cc:407] Connection header received
0807 15:14:06.592655 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:14:06.592656 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:14:06.592668 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:14:06.592696 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:14:06.593567 (+ 871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:06.594290 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:06.595382 (+ 1092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:06.595702 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:06.595749 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:06.597561 (+ 1812us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:14:06.597563 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:14:06.597563 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:14:06.597570 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:14:06.599373 (+ 1803us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:14:06.599922 (+ 549us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:14:06.599924 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:14:06.599924 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:14:06.599984 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:14:06.600371 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:14:06.600372 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:14:06.600373 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:14:06.600565 (+ 192us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:14:06.600672 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0807 15:14:06.600900 (+ 228us) server_negotiation.cc:299] Negotiation successful
0807 15:14:06.600994 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260807 15:14:06.601590 17930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:06.591854 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40248 (local address 127.29.194.62:33665)
0807 15:14:06.591898 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0807 15:14:06.591900 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:14:06.592555 (+ 655us) server_negotiation.cc:407] Connection header received
0807 15:14:06.592659 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:14:06.592660 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:14:06.592670 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:14:06.592699 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:14:06.593611 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:06.594346 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:06.595402 (+ 1056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:06.595690 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:06.595732 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:06.597551 (+ 1819us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:14:06.597553 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:14:06.597553 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:14:06.597561 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:14:06.599539 (+ 1978us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:14:06.600131 (+ 592us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:14:06.600132 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:14:06.600133 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:14:06.600255 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:14:06.600468 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:14:06.600469 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:14:06.600469 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:14:06.600631 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:14:06.600801 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0807 15:14:06.601377 (+ 576us) server_negotiation.cc:299] Negotiation successful
0807 15:14:06.601500 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260807 15:14:06.602746 18050 master_runner.cc:186] Error getting master registration for 127.29.194.61:38713: OK, instance_id { permanent_uuid: "b978bb5878a244428c9c03f7d108395a" instance_seqno: 1786115646384010 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260807 15:14:06.605196 18050 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260807 15:14:06.607753 18046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:06.592290 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59148 (local address 127.29.194.61:38713)
0807 15:14:06.592358 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0807 15:14:06.592359 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 15:14:06.600844 (+ 8485us) server_negotiation.cc:407] Connection header received
0807 15:14:06.600946 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:14:06.600948 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:14:06.600960 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:14:06.600991 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:14:06.601659 (+ 668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:06.602321 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:06.603121 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:06.603389 (+ 268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:06.603428 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:06.604966 (+ 1538us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:14:06.604967 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:14:06.604968 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:14:06.604975 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:14:06.606445 (+ 1470us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:14:06.606832 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:14:06.606833 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:14:06.606834 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:14:06.607031 (+ 197us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:14:06.607254 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:14:06.607255 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:14:06.607255 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:14:06.607369 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:14:06.607470 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0807 15:14:06.607582 (+ 112us) server_negotiation.cc:299] Negotiation successful
0807 15:14:06.607663 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260807 15:14:06.620631 18037 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.194.60" port: 42169 } attempt: 3
Aug 07 15:14:06 dist-test-slave-42z9 krb5kdc[17779](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115646, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.194.61@KRBTEST.COM for kudu/127.29.194.60@KRBTEST.COM
I20260807 15:14:06.628556 18039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:06.621031 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.60:42169 (local address 127.0.0.1:59848)
0807 15:14:06.621096 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0807 15:14:06.621109 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 15:14:06.621146 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:14:06.621283 (+ 137us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:14:06.621284 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:14:06.621489 (+ 205us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:14:06.621544 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:14:06.621546 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:14:06.622359 (+ 813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:14:06.622361 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:14:06.623000 (+ 639us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:14:06.623003 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:14:06.623339 (+ 336us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:14:06.623340 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:14:06.623400 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:06.623822 (+ 422us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:14:06.623828 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:14:06.625644 (+ 1816us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:14:06.627403 (+ 1759us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:14:06.627405 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:14:06.627405 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:14:06.627610 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:14:06.627792 (+ 182us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:14:06.627793 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:14:06.627793 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:14:06.627857 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:14:06.628176 (+ 319us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:14:06.628177 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:14:06.628238 (+ 61us) client_negotiation.cc:769] Sending connection context
0807 15:14:06.628301 (+ 63us) client_negotiation.cc:240] Negotiation successful
0807 15:14:06.628440 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43}
I20260807 15:14:06.629288 18161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:06.621080 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59848 (local address 127.29.194.60:42169)
0807 15:14:06.621120 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0807 15:14:06.621122 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:14:06.621150 (+ 28us) server_negotiation.cc:407] Connection header received
0807 15:14:06.621217 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:14:06.621217 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:14:06.621228 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:14:06.621271 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:14:06.621611 (+ 340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:06.622268 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:06.623078 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:06.623279 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:06.623334 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:06.625787 (+ 2453us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:14:06.625790 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:14:06.625790 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:14:06.625801 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:14:06.627297 (+ 1496us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:14:06.627678 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:14:06.627680 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:14:06.627680 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:14:06.627736 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:14:06.627908 (+ 172us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:14:06.627909 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:14:06.627909 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:14:06.628066 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:14:06.628131 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0807 15:14:06.629081 (+ 950us) server_negotiation.cc:299] Negotiation successful
0807 15:14:06.629194 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260807 15:14:06.631590 18037 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b978bb5878a244428c9c03f7d108395a: Bootstrap starting.
I20260807 15:14:06.632375 18037 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b978bb5878a244428c9c03f7d108395a: Neither blocks nor log segments found. Creating new log.
I20260807 15:14:06.632638 18037 log.cc:824] T 00000000000000000000000000000000 P b978bb5878a244428c9c03f7d108395a: Log is configured to *not* fsync() on all Append() calls
I20260807 15:14:06.633510 18037 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b978bb5878a244428c9c03f7d108395a: No bootstrap required, opened a new log
I20260807 15:14:06.635825 18037 raft_consensus.cc:348] T 00000000000000000000000000000000 P b978bb5878a244428c9c03f7d108395a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f1ebc2f7a7a40a598a4c98d85e350ab" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 33665 } } peers { permanent_uuid: "b978bb5878a244428c9c03f7d108395a" member_type: VOTER last_known_addr { host: "127.29.194.61" port: 38713 } } peers { permanent_uuid: "28c75a9c172140c4b412aaf9629fabf1" member_type: VOTER last_known_addr { host: "127.29.194.60" port: 42169 } }
I20260807 15:14:06.635952 18037 raft_consensus.cc:374] T 00000000000000000000000000000000 P b978bb5878a244428c9c03f7d108395a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:14:06.635986 18037 raft_consensus.cc:729] T 00000000000000000000000000000000 P b978bb5878a244428c9c03f7d108395a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b978bb5878a244428c9c03f7d108395a, State: Initialized, Role: FOLLOWER
I20260807 15:14:06.636139 18037 consensus_queue.cc:260] T 00000000000000000000000000000000 P b978bb5878a244428c9c03f7d108395a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f1ebc2f7a7a40a598a4c98d85e350ab" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 33665 } } peers { permanent_uuid: "b978bb5878a244428c9c03f7d108395a" member_type: VOTER last_known_addr { host: "127.29.194.61" port: 38713 } } peers { permanent_uuid: "28c75a9c172140c4b412aaf9629fabf1" member_type: VOTER last_known_addr { host: "127.29.194.60" port: 42169 } }
I20260807 15:14:06.636412 18211 sys_catalog.cc:455] T 00000000000000000000000000000000 P b978bb5878a244428c9c03f7d108395a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f1ebc2f7a7a40a598a4c98d85e350ab" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 33665 } } peers { permanent_uuid: "b978bb5878a244428c9c03f7d108395a" member_type: VOTER last_known_addr { host: "127.29.194.61" port: 38713 } } peers { permanent_uuid: "28c75a9c172140c4b412aaf9629fabf1" member_type: VOTER last_known_addr { host: "127.29.194.60" port: 42169 } } }
I20260807 15:14:06.636488 18037 sys_catalog.cc:564] T 00000000000000000000000000000000 P b978bb5878a244428c9c03f7d108395a [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:14:06.636566 18211 sys_catalog.cc:458] T 00000000000000000000000000000000 P b978bb5878a244428c9c03f7d108395a [sys.catalog]: This master's current role is: FOLLOWER
W20260807 15:14:06.638399 18226 catalog_manager.cc:1558] T 00000000000000000000000000000000 P b978bb5878a244428c9c03f7d108395a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 15:14:06.638517 18226 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 15:14:06.640481 17933 master_runner.cc:263] No leader master found from master b978bb5878a244428c9c03f7d108395a
I20260807 15:14:06.640558 17933 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260807 15:14:06.669607 18164 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1f1ebc2f7a7a40a598a4c98d85e350ab [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 15:14:06.669755 18164 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1f1ebc2f7a7a40a598a4c98d85e350ab [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f1ebc2f7a7a40a598a4c98d85e350ab" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 33665 } } peers { permanent_uuid: "b978bb5878a244428c9c03f7d108395a" member_type: VOTER last_known_addr { host: "127.29.194.61" port: 38713 } } peers { permanent_uuid: "28c75a9c172140c4b412aaf9629fabf1" member_type: VOTER last_known_addr { host: "127.29.194.60" port: 42169 } }
I20260807 15:14:06.670044 18164 leader_election.cc:288] T 00000000000000000000000000000000 P 1f1ebc2f7a7a40a598a4c98d85e350ab [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b978bb5878a244428c9c03f7d108395a (127.29.194.61:38713), 28c75a9c172140c4b412aaf9629fabf1 (127.29.194.60:42169)
I20260807 15:14:06.670610 17992 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "1f1ebc2f7a7a40a598a4c98d85e350ab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b978bb5878a244428c9c03f7d108395a" is_pre_election: true
I20260807 15:14:06.670804 17992 raft_consensus.cc:2454] T 00000000000000000000000000000000 P b978bb5878a244428c9c03f7d108395a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f1ebc2f7a7a40a598a4c98d85e350ab in term 0.
I20260807 15:14:06.671125 17838 leader_election.cc:302] T 00000000000000000000000000000000 P 1f1ebc2f7a7a40a598a4c98d85e350ab [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f1ebc2f7a7a40a598a4c98d85e350ab, b978bb5878a244428c9c03f7d108395a; no voters:
I20260807 15:14:06.671602 18119 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "1f1ebc2f7a7a40a598a4c98d85e350ab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28c75a9c172140c4b412aaf9629fabf1" is_pre_election: true
I20260807 15:14:06.671762 18119 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 28c75a9c172140c4b412aaf9629fabf1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f1ebc2f7a7a40a598a4c98d85e350ab in term 0.
I20260807 15:14:06.672091 18164 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1f1ebc2f7a7a40a598a4c98d85e350ab [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 15:14:06.672147 18164 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1f1ebc2f7a7a40a598a4c98d85e350ab [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 15:14:06.672168 18164 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1f1ebc2f7a7a40a598a4c98d85e350ab [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:14:06.673075 18164 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1f1ebc2f7a7a40a598a4c98d85e350ab [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f1ebc2f7a7a40a598a4c98d85e350ab" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 33665 } } peers { permanent_uuid: "b978bb5878a244428c9c03f7d108395a" member_type: VOTER last_known_addr { host: "127.29.194.61" port: 38713 } } peers { permanent_uuid: "28c75a9c172140c4b412aaf9629fabf1" member_type: VOTER last_known_addr { host: "127.29.194.60" port: 42169 } }
I20260807 15:14:06.673213 18164 leader_election.cc:288] T 00000000000000000000000000000000 P 1f1ebc2f7a7a40a598a4c98d85e350ab [CANDIDATE]: Term 1 election: Requested vote from peers b978bb5878a244428c9c03f7d108395a (127.29.194.61:38713), 28c75a9c172140c4b412aaf9629fabf1 (127.29.194.60:42169)
I20260807 15:14:06.673568 17992 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "1f1ebc2f7a7a40a598a4c98d85e350ab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b978bb5878a244428c9c03f7d108395a"
I20260807 15:14:06.673653 17992 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b978bb5878a244428c9c03f7d108395a [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:14:06.674633 17992 raft_consensus.cc:2454] T 00000000000000000000000000000000 P b978bb5878a244428c9c03f7d108395a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f1ebc2f7a7a40a598a4c98d85e350ab in term 1.
I20260807 15:14:06.674841 17838 leader_election.cc:302] T 00000000000000000000000000000000 P 1f1ebc2f7a7a40a598a4c98d85e350ab [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f1ebc2f7a7a40a598a4c98d85e350ab, b978bb5878a244428c9c03f7d108395a; no voters:
I20260807 15:14:06.675100 18119 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "1f1ebc2f7a7a40a598a4c98d85e350ab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28c75a9c172140c4b412aaf9629fabf1"
I20260807 15:14:06.675189 18119 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 28c75a9c172140c4b412aaf9629fabf1 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:14:06.676059 18119 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 28c75a9c172140c4b412aaf9629fabf1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f1ebc2f7a7a40a598a4c98d85e350ab in term 1.
I20260807 15:14:06.677295 18164 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1f1ebc2f7a7a40a598a4c98d85e350ab [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:14:06.677501 18164 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1f1ebc2f7a7a40a598a4c98d85e350ab [term 1 LEADER]: Becoming Leader. State: Replica: 1f1ebc2f7a7a40a598a4c98d85e350ab, State: Running, Role: LEADER
I20260807 15:14:06.677606 18164 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1f1ebc2f7a7a40a598a4c98d85e350ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f1ebc2f7a7a40a598a4c98d85e350ab" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 33665 } } peers { permanent_uuid: "b978bb5878a244428c9c03f7d108395a" member_type: VOTER last_known_addr { host: "127.29.194.61" port: 38713 } } peers { permanent_uuid: "28c75a9c172140c4b412aaf9629fabf1" member_type: VOTER last_known_addr { host: "127.29.194.60" port: 42169 } }
I20260807 15:14:06.677994 18164 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1f1ebc2f7a7a40a598a4c98d85e350ab [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1f1ebc2f7a7a40a598a4c98d85e350ab. Latest consensus state: current_term: 1 leader_uuid: "1f1ebc2f7a7a40a598a4c98d85e350ab" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f1ebc2f7a7a40a598a4c98d85e350ab" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 33665 } } peers { permanent_uuid: "b978bb5878a244428c9c03f7d108395a" member_type: VOTER last_known_addr { host: "127.29.194.61" port: 38713 } } peers { permanent_uuid: "28c75a9c172140c4b412aaf9629fabf1" member_type: VOTER last_known_addr { host: "127.29.194.60" port: 42169 } } }
I20260807 15:14:06.678054 18164 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1f1ebc2f7a7a40a598a4c98d85e350ab [sys.catalog]: This master's current role is: LEADER
I20260807 15:14:06.678529 18231 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:14:06.678820 18231 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:14:06.680980 18119 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 28c75a9c172140c4b412aaf9629fabf1 [term 1 FOLLOWER]: Refusing update from remote peer 1f1ebc2f7a7a40a598a4c98d85e350ab: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:14:06.681392 18164 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 1f1ebc2f7a7a40a598a4c98d85e350ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "28c75a9c172140c4b412aaf9629fabf1" member_type: VOTER last_known_addr { host: "127.29.194.60" port: 42169 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:14:06.682360 17992 raft_consensus.cc:1261] T 00000000000000000000000000000000 P b978bb5878a244428c9c03f7d108395a [term 1 FOLLOWER]: Refusing update from remote peer 1f1ebc2f7a7a40a598a4c98d85e350ab: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:14:06.682708 18164 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 1f1ebc2f7a7a40a598a4c98d85e350ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "b978bb5878a244428c9c03f7d108395a" member_type: VOTER last_known_addr { host: "127.29.194.61" port: 38713 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:14:06.683210 18193 sys_catalog.cc:455] T 00000000000000000000000000000000 P 28c75a9c172140c4b412aaf9629fabf1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1f1ebc2f7a7a40a598a4c98d85e350ab. Latest consensus state: current_term: 1 leader_uuid: "1f1ebc2f7a7a40a598a4c98d85e350ab" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f1ebc2f7a7a40a598a4c98d85e350ab" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 33665 } } peers { permanent_uuid: "b978bb5878a244428c9c03f7d108395a" member_type: VOTER last_known_addr { host: "127.29.194.61" port: 38713 } } peers { permanent_uuid: "28c75a9c172140c4b412aaf9629fabf1" member_type: VOTER last_known_addr { host: "127.29.194.60" port: 42169 } } }
I20260807 15:14:06.683310 18193 sys_catalog.cc:458] T 00000000000000000000000000000000 P 28c75a9c172140c4b412aaf9629fabf1 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 15:14:06.683877 18211 sys_catalog.cc:455] T 00000000000000000000000000000000 P b978bb5878a244428c9c03f7d108395a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1f1ebc2f7a7a40a598a4c98d85e350ab. Latest consensus state: current_term: 1 leader_uuid: "1f1ebc2f7a7a40a598a4c98d85e350ab" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f1ebc2f7a7a40a598a4c98d85e350ab" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 33665 } } peers { permanent_uuid: "b978bb5878a244428c9c03f7d108395a" member_type: VOTER last_known_addr { host: "127.29.194.61" port: 38713 } } peers { permanent_uuid: "28c75a9c172140c4b412aaf9629fabf1" member_type: VOTER last_known_addr { host: "127.29.194.60" port: 42169 } } }
I20260807 15:14:06.683969 18211 sys_catalog.cc:458] T 00000000000000000000000000000000 P b978bb5878a244428c9c03f7d108395a [sys.catalog]: This master's current role is: FOLLOWER
I20260807 15:14:06.685053 18164 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1f1ebc2f7a7a40a598a4c98d85e350ab [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "1f1ebc2f7a7a40a598a4c98d85e350ab" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f1ebc2f7a7a40a598a4c98d85e350ab" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 33665 } } peers { permanent_uuid: "b978bb5878a244428c9c03f7d108395a" member_type: VOTER last_known_addr { host: "127.29.194.61" port: 38713 } } peers { permanent_uuid: "28c75a9c172140c4b412aaf9629fabf1" member_type: VOTER last_known_addr { host: "127.29.194.60" port: 42169 } } }
I20260807 15:14:06.685138 18164 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1f1ebc2f7a7a40a598a4c98d85e350ab [sys.catalog]: This master's current role is: LEADER
I20260807 15:14:06.685633 18211 sys_catalog.cc:455] T 00000000000000000000000000000000 P b978bb5878a244428c9c03f7d108395a [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "1f1ebc2f7a7a40a598a4c98d85e350ab" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f1ebc2f7a7a40a598a4c98d85e350ab" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 33665 } } peers { permanent_uuid: "b978bb5878a244428c9c03f7d108395a" member_type: VOTER last_known_addr { host: "127.29.194.61" port: 38713 } } peers { permanent_uuid: "28c75a9c172140c4b412aaf9629fabf1" member_type: VOTER last_known_addr { host: "127.29.194.60" port: 42169 } } }
I20260807 15:14:06.685732 18211 sys_catalog.cc:458] T 00000000000000000000000000000000 P b978bb5878a244428c9c03f7d108395a [sys.catalog]: This master's current role is: FOLLOWER
I20260807 15:14:06.685962 18235 mvcc.cc:205] Tried to move back new op lower bound from 7315929688802631680 to 7315929688791961600. Current Snapshot: MvccSnapshot[applied={T|T < 7315929688802631680}]
I20260807 15:14:06.686594 18240 mvcc.cc:205] Tried to move back new op lower bound from 7315929688802631680 to 7315929688791961600. Current Snapshot: MvccSnapshot[applied={T|T < 7315929688802631680}]
I20260807 15:14:06.687176 18231 catalog_manager.cc:1347] Generated new cluster ID: e41be71ba6374a2895c34b88406f4222
I20260807 15:14:06.687220 18231 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:14:06.690385 18164 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1f1ebc2f7a7a40a598a4c98d85e350ab [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "1f1ebc2f7a7a40a598a4c98d85e350ab" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f1ebc2f7a7a40a598a4c98d85e350ab" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 33665 } } peers { permanent_uuid: "b978bb5878a244428c9c03f7d108395a" member_type: VOTER last_known_addr { host: "127.29.194.61" port: 38713 } } peers { permanent_uuid: "28c75a9c172140c4b412aaf9629fabf1" member_type: VOTER last_known_addr { host: "127.29.194.60" port: 42169 } } }
I20260807 15:14:06.690513 18164 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1f1ebc2f7a7a40a598a4c98d85e350ab [sys.catalog]: This master's current role is: LEADER
I20260807 15:14:06.690909 18193 sys_catalog.cc:455] T 00000000000000000000000000000000 P 28c75a9c172140c4b412aaf9629fabf1 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "1f1ebc2f7a7a40a598a4c98d85e350ab" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f1ebc2f7a7a40a598a4c98d85e350ab" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 33665 } } peers { permanent_uuid: "b978bb5878a244428c9c03f7d108395a" member_type: VOTER last_known_addr { host: "127.29.194.61" port: 38713 } } peers { permanent_uuid: "28c75a9c172140c4b412aaf9629fabf1" member_type: VOTER last_known_addr { host: "127.29.194.60" port: 42169 } } }
I20260807 15:14:06.690992 18193 sys_catalog.cc:458] T 00000000000000000000000000000000 P 28c75a9c172140c4b412aaf9629fabf1 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 15:14:06.691560 18236 mvcc.cc:205] Tried to move back new op lower bound from 7315929688802631680 to 7315929688791961600. Current Snapshot: MvccSnapshot[applied={T|T < 7315929688802631680}]
I20260807 15:14:06.695756 18231 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:14:06.696504 18231 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:14:06.710222 18231 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1f1ebc2f7a7a40a598a4c98d85e350ab: Generated new TSK 0
I20260807 15:14:06.710520 18231 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 15:14:07.601038 18207 catalog_manager.cc:1259] Loaded cluster ID: e41be71ba6374a2895c34b88406f4222
I20260807 15:14:07.601127 18207 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 28c75a9c172140c4b412aaf9629fabf1: loading cluster ID for follower catalog manager: success
I20260807 15:14:07.602111 18207 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 28c75a9c172140c4b412aaf9629fabf1: acquiring CA information for follower catalog manager: success
I20260807 15:14:07.602447 18207 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 28c75a9c172140c4b412aaf9629fabf1: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 15:14:07.639060 18226 catalog_manager.cc:1259] Loaded cluster ID: e41be71ba6374a2895c34b88406f4222
I20260807 15:14:07.639154 18226 catalog_manager.cc:1552] T 00000000000000000000000000000000 P b978bb5878a244428c9c03f7d108395a: loading cluster ID for follower catalog manager: success
I20260807 15:14:07.640009 18226 catalog_manager.cc:1574] T 00000000000000000000000000000000 P b978bb5878a244428c9c03f7d108395a: acquiring CA information for follower catalog manager: success
I20260807 15:14:07.640331 18226 catalog_manager.cc:1602] T 00000000000000000000000000000000 P b978bb5878a244428c9c03f7d108395a: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 15:14:07.770102 30472 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7fZSIL/build/release/bin/kudu with pid 17801
I20260807 15:14:07.788450 30472 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7fZSIL/build/release/bin/kudu with pid 17933
I20260807 15:14:07.800797 30472 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7fZSIL/build/release/bin/kudu with pid 18050
2026-08-07T15:14:07Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (2986 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786115480851507-30472-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 15:14:07 dist-test-slave-42z9 krb5kdc[18265](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 15:14:07 dist-test-slave-42z9 krb5kdc[18265](info): set up 2 sockets
Aug 07 15:14:07 dist-test-slave-42z9 krb5kdc[18265](info): commencing operation
krb5kdc: starting...
W20260807 15:14:08.951565 30472 mini_kdc.cc:121] Time spent starting KDC: real 1.123s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786115480851507-30472-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786115480851507-30472-0/krb5kdc/test-user.keytab.
Aug 07 15:14:09 dist-test-slave-42z9 krb5kdc[18265](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115649, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T15:14:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T15:14:09Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.194.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.194.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.194.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.62.keytab.
Entry for principal kudu/127.29.194.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.62.keytab.
I20260807 15:14:09.089723 30472 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7fZSIL/build/release/bin/kudu
/tmp/dist-test-task7fZSIL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786115480851507-30472-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786115480851507-30472-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786115480851507-30472-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786115480851507-30472-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.194.62:40139
--webserver_interface=127.29.194.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.62.keytab
--principal=kudu/127.29.194.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.194.20:46773
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.194.62:40139
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786115480851507-30472-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786115480851507-30472-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786115480851507-30472-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:14:09.180471 18289 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:14:09.180729 18289 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:14:09.180754 18289 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:14:09.182097 18289 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:14:09.182157 18289 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:14:09.182171 18289 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:14:09.182183 18289 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:14:09.182194 18289 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:14:09.183921 18289 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.194.20:46773
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786115480851507-30472-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786115480851507-30472-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.194.62:40139
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.62.keytab
--principal=kudu/127.29.194.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.194.62:40139
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786115480851507-30472-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.194.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18289
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786115480851507-30472-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:14:09.184207 18289 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:14:09.184499 18289 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:14:09.190265 18298 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:14:09.190456 18299 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:14:09.190295 18289 server_base.cc:1034] running on GCE node
W20260807 15:14:09.190631 18301 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:14:09.191049 18289 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:14:09.191370 18289 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:14:09.192554 18289 hybrid_clock.cc:648] HybridClock initialized: now 1786115649192532 us; error 44 us; skew 500 ppm
Aug 07 15:14:09 dist-test-slave-42z9 krb5kdc[18265](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115649, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.194.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:14:09.195202 18289 init.cc:377] Logged in from keytab as kudu/127.29.194.62@KRBTEST.COM (short username kudu)
I20260807 15:14:09.195708 18289 webserver.cc:460] Webserver started at http://127.29.194.62:44505/ using document root <none> and password file <none>
I20260807 15:14:09.195955 18289 fs_manager.cc:359] Metadata directory not provided
I20260807 15:14:09.196012 18289 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:14:09.196149 18289 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:14:09.197244 18289 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786115480851507-30472-0/minicluster-data/master-0/data/instance:
uuid: "7361c4fd2516462795ceb761bde24078"
format_stamp: "Formatted at 2026-08-07 15:14:09 on dist-test-slave-42z9"
server_key: "c409e1921abceb4e93adcc1bdb1ada27"
server_key_iv: "28c5a110269636567d6fe11085d5d6f3"
server_key_version: "encryptionkey@0"
I20260807 15:14:09.197618 18289 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786115480851507-30472-0/minicluster-data/master-0/wal/instance:
uuid: "7361c4fd2516462795ceb761bde24078"
format_stamp: "Formatted at 2026-08-07 15:14:09 on dist-test-slave-42z9"
server_key: "c409e1921abceb4e93adcc1bdb1ada27"
server_key_iv: "28c5a110269636567d6fe11085d5d6f3"
server_key_version: "encryptionkey@0"
I20260807 15:14:09.199002 18289 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260807 15:14:09.199843 18310 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:14:09.200058 18289 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 15:14:09.200161 18289 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786115480851507-30472-0/minicluster-data/master-0/data,/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786115480851507-30472-0/minicluster-data/master-0/wal
uuid: "7361c4fd2516462795ceb761bde24078"
format_stamp: "Formatted at 2026-08-07 15:14:09 on dist-test-slave-42z9"
server_key: "c409e1921abceb4e93adcc1bdb1ada27"
server_key_iv: "28c5a110269636567d6fe11085d5d6f3"
server_key_version: "encryptionkey@0"
I20260807 15:14:09.200237 18289 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786115480851507-30472-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786115480851507-30472-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786115480851507-30472-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:14:09.217701 18289 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:14:09.220575 18289 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:14:09.220758 18289 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:14:09.225019 18289 rpc_server.cc:307] RPC server started. Bound to: 127.29.194.62:40139
I20260807 15:14:09.225069 18385 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.194.62:40139 every 8 connection(s)
I20260807 15:14:09.225428 18289 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786115480851507-30472-0/minicluster-data/master-0/data/info.pb
I20260807 15:14:09.226409 30472 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7fZSIL/build/release/bin/kudu as pid 18289
I20260807 15:14:09.226509 30472 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786115480851507-30472-0/minicluster-data/master-0/wal/instance
I20260807 15:14:09.226529 18386 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:14:09.226720 30472 external_mini_cluster.cc:1428] Setting key ee23cbb83096c164b987e631f130f00d
I20260807 15:14:09.229714 18386 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7361c4fd2516462795ceb761bde24078: Bootstrap starting.
I20260807 15:14:09.230422 18386 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7361c4fd2516462795ceb761bde24078: Neither blocks nor log segments found. Creating new log.
I20260807 15:14:09.230728 18386 log.cc:824] T 00000000000000000000000000000000 P 7361c4fd2516462795ceb761bde24078: Log is configured to *not* fsync() on all Append() calls
I20260807 15:14:09.231932 18386 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7361c4fd2516462795ceb761bde24078: No bootstrap required, opened a new log
Aug 07 15:14:09 dist-test-slave-42z9 krb5kdc[18265](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115649, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.194.62@KRBTEST.COM
I20260807 15:14:09.234537 18386 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7361c4fd2516462795ceb761bde24078 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7361c4fd2516462795ceb761bde24078" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 40139 } }
I20260807 15:14:09.234683 18386 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7361c4fd2516462795ceb761bde24078 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:14:09.234704 18386 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7361c4fd2516462795ceb761bde24078 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7361c4fd2516462795ceb761bde24078, State: Initialized, Role: FOLLOWER
I20260807 15:14:09.234829 18386 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7361c4fd2516462795ceb761bde24078 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7361c4fd2516462795ceb761bde24078" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 40139 } }
I20260807 15:14:09.234905 18386 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7361c4fd2516462795ceb761bde24078 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:14:09.234927 18386 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7361c4fd2516462795ceb761bde24078 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:14:09.234972 18386 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7361c4fd2516462795ceb761bde24078 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:14:09.235656 18386 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7361c4fd2516462795ceb761bde24078 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7361c4fd2516462795ceb761bde24078" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 40139 } }
I20260807 15:14:09.235765 18386 leader_election.cc:302] T 00000000000000000000000000000000 P 7361c4fd2516462795ceb761bde24078 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7361c4fd2516462795ceb761bde24078; no voters:
I20260807 15:14:09.235954 18386 leader_election.cc:288] T 00000000000000000000000000000000 P 7361c4fd2516462795ceb761bde24078 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 15:14:09.236003 18392 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7361c4fd2516462795ceb761bde24078 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:14:09.236169 18392 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7361c4fd2516462795ceb761bde24078 [term 1 LEADER]: Becoming Leader. State: Replica: 7361c4fd2516462795ceb761bde24078, State: Running, Role: LEADER
I20260807 15:14:09.236260 18392 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7361c4fd2516462795ceb761bde24078 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7361c4fd2516462795ceb761bde24078" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 40139 } }
I20260807 15:14:09.236265 18386 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7361c4fd2516462795ceb761bde24078 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:14:09.237754 18394 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7361c4fd2516462795ceb761bde24078 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7361c4fd2516462795ceb761bde24078" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7361c4fd2516462795ceb761bde24078" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 40139 } } }
I20260807 15:14:09.237862 18394 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7361c4fd2516462795ceb761bde24078 [sys.catalog]: This master's current role is: LEADER
I20260807 15:14:09.238018 18396 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7361c4fd2516462795ceb761bde24078 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7361c4fd2516462795ceb761bde24078. Latest consensus state: current_term: 1 leader_uuid: "7361c4fd2516462795ceb761bde24078" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7361c4fd2516462795ceb761bde24078" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 40139 } } }
I20260807 15:14:09.238086 18396 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7361c4fd2516462795ceb761bde24078 [sys.catalog]: This master's current role is: LEADER
W20260807 15:14:09.238262 18414 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 7361c4fd2516462795ceb761bde24078: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 15:14:09.238309 18414 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 15:14:09.238433 18415 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:14:09.238598 18390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:09.227288 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40630 (local address 127.29.194.62:40139)
0807 15:14:09.227472 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0807 15:14:09.227479 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 15:14:09.227504 (+ 25us) server_negotiation.cc:407] Connection header received
0807 15:14:09.227969 (+ 465us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:14:09.227976 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:14:09.228024 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:14:09.228138 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:14:09.228973 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:09.230041 (+ 1068us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:09.231079 (+ 1038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:09.231647 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:09.231774 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:09.234682 (+ 2908us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:14:09.234687 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:14:09.234687 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:14:09.234701 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:14:09.236606 (+ 1905us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:14:09.237044 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:14:09.237047 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:14:09.237048 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:14:09.237148 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:14:09.237371 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:14:09.237372 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:14:09.237372 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:14:09.237781 (+ 409us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:14:09.238005 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0807 15:14:09.238193 (+ 188us) server_negotiation.cc:299] Negotiation successful
0807 15:14:09.238336 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":139,"thread_start_us":69,"threads_started":1}
I20260807 15:14:09.238658 18415 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:14:09.240017 30472 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 15:14:09.241880 18415 catalog_manager.cc:1347] Generated new cluster ID: faa1cb62edab4067b7ce88441341de44
I20260807 15:14:09.241942 18415 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:14:09.264071 18415 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:14:09.264815 18415 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:14:09.270517 18415 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7361c4fd2516462795ceb761bde24078: Generated new TSK 0
I20260807 15:14:09.270740 18415 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 15:14:09.293969 30472 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7fZSIL/build/release/bin/kudu with pid 18289
2026-08-07T15:14:09Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1492 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115480851507-30472-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 15:14:09 dist-test-slave-42z9 krb5kdc[18425](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 15:14:09 dist-test-slave-42z9 krb5kdc[18425](info): set up 2 sockets
Aug 07 15:14:09 dist-test-slave-42z9 krb5kdc[18425](info): commencing operation
krb5kdc: starting...
W20260807 15:14:10.442276 30472 mini_kdc.cc:121] Time spent starting KDC: real 1.121s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115480851507-30472-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115480851507-30472-0/krb5kdc/test-user.keytab.
Aug 07 15:14:10 dist-test-slave-42z9 krb5kdc[18425](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115650, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T15:14:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T15:14:10Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.194.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.194.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.194.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.62.keytab.
Entry for principal kudu/127.29.194.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.62.keytab.
I20260807 15:14:10.584630 30472 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7fZSIL/build/release/bin/kudu
/tmp/dist-test-task7fZSIL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115480851507-30472-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115480851507-30472-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115480851507-30472-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115480851507-30472-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.194.62:46167
--webserver_interface=127.29.194.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.62.keytab
--principal=kudu/127.29.194.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.194.20:43617
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.194.62:46167
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115480851507-30472-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115480851507-30472-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115480851507-30472-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:14:10.673857 18445 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:14:10.674105 18445 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:14:10.674131 18445 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:14:10.675377 18445 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:14:10.675436 18445 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:14:10.675449 18445 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:14:10.675460 18445 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:14:10.675475 18445 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:14:10.677107 18445 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.194.20:43617
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115480851507-30472-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115480851507-30472-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.194.62:46167
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.62.keytab
--principal=kudu/127.29.194.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.194.62:46167
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115480851507-30472-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.194.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18445
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115480851507-30472-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:14:10.677382 18445 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:14:10.677796 18445 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:14:10.684310 18455 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:14:10.684310 18454 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:14:10.684312 18458 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:14:10.684388 18445 server_base.cc:1034] running on GCE node
I20260807 15:14:10.684857 18445 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:14:10.685163 18445 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:14:10.686393 18445 hybrid_clock.cc:648] HybridClock initialized: now 1786115650686366 us; error 31 us; skew 500 ppm
Aug 07 15:14:10 dist-test-slave-42z9 krb5kdc[18425](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115650, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.194.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:14:10.689765 18445 init.cc:377] Logged in from keytab as kudu/127.29.194.62@KRBTEST.COM (short username kudu)
I20260807 15:14:10.690369 18445 webserver.cc:460] Webserver started at http://127.29.194.62:42443/ using document root <none> and password file <none>
I20260807 15:14:10.690629 18445 fs_manager.cc:359] Metadata directory not provided
I20260807 15:14:10.690686 18445 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:14:10.690845 18445 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:14:10.692374 18445 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115480851507-30472-0/minicluster-data/master-0/data/instance:
uuid: "25474f26b3ab4143b43529dc11548fe2"
format_stamp: "Formatted at 2026-08-07 15:14:10 on dist-test-slave-42z9"
server_key: "5499c9bcc5ff9f5b2103e1aa0425b153"
server_key_iv: "12cdb2959cfffadfd57e8f3f376805c6"
server_key_version: "encryptionkey@0"
I20260807 15:14:10.692833 18445 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115480851507-30472-0/minicluster-data/master-0/wal/instance:
uuid: "25474f26b3ab4143b43529dc11548fe2"
format_stamp: "Formatted at 2026-08-07 15:14:10 on dist-test-slave-42z9"
server_key: "5499c9bcc5ff9f5b2103e1aa0425b153"
server_key_iv: "12cdb2959cfffadfd57e8f3f376805c6"
server_key_version: "encryptionkey@0"
I20260807 15:14:10.694482 18445 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.004s sys 0.000s
I20260807 15:14:10.695490 18470 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:14:10.695711 18445 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:14:10.695804 18445 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115480851507-30472-0/minicluster-data/master-0/data,/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115480851507-30472-0/minicluster-data/master-0/wal
uuid: "25474f26b3ab4143b43529dc11548fe2"
format_stamp: "Formatted at 2026-08-07 15:14:10 on dist-test-slave-42z9"
server_key: "5499c9bcc5ff9f5b2103e1aa0425b153"
server_key_iv: "12cdb2959cfffadfd57e8f3f376805c6"
server_key_version: "encryptionkey@0"
I20260807 15:14:10.695879 18445 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115480851507-30472-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115480851507-30472-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115480851507-30472-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:14:10.712087 18445 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:14:10.714313 18445 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:14:10.714459 18445 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:14:10.718677 18445 rpc_server.cc:307] RPC server started. Bound to: 127.29.194.62:46167
I20260807 15:14:10.718808 18543 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.194.62:46167 every 8 connection(s)
I20260807 15:14:10.719117 18445 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115480851507-30472-0/minicluster-data/master-0/data/info.pb
I20260807 15:14:10.720201 18544 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:14:10.721136 30472 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7fZSIL/build/release/bin/kudu as pid 18445
I20260807 15:14:10.721246 30472 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115480851507-30472-0/minicluster-data/master-0/wal/instance
I20260807 15:14:10.721412 30472 external_mini_cluster.cc:1428] Setting key 7eb3e396efd5b5710b29cb802e0f9b79
I20260807 15:14:10.722931 18544 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 25474f26b3ab4143b43529dc11548fe2: Bootstrap starting.
I20260807 15:14:10.723729 18544 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 25474f26b3ab4143b43529dc11548fe2: Neither blocks nor log segments found. Creating new log.
I20260807 15:14:10.724032 18544 log.cc:824] T 00000000000000000000000000000000 P 25474f26b3ab4143b43529dc11548fe2: Log is configured to *not* fsync() on all Append() calls
I20260807 15:14:10.724944 18544 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 25474f26b3ab4143b43529dc11548fe2: No bootstrap required, opened a new log
I20260807 15:14:10.727327 18544 raft_consensus.cc:348] T 00000000000000000000000000000000 P 25474f26b3ab4143b43529dc11548fe2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25474f26b3ab4143b43529dc11548fe2" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 46167 } }
I20260807 15:14:10.727540 18544 raft_consensus.cc:374] T 00000000000000000000000000000000 P 25474f26b3ab4143b43529dc11548fe2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:14:10.727593 18544 raft_consensus.cc:729] T 00000000000000000000000000000000 P 25474f26b3ab4143b43529dc11548fe2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 25474f26b3ab4143b43529dc11548fe2, State: Initialized, Role: FOLLOWER
I20260807 15:14:10.727739 18544 consensus_queue.cc:260] T 00000000000000000000000000000000 P 25474f26b3ab4143b43529dc11548fe2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25474f26b3ab4143b43529dc11548fe2" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 46167 } }
I20260807 15:14:10.727844 18544 raft_consensus.cc:388] T 00000000000000000000000000000000 P 25474f26b3ab4143b43529dc11548fe2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:14:10.727880 18544 raft_consensus.cc:482] T 00000000000000000000000000000000 P 25474f26b3ab4143b43529dc11548fe2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:14:10.727922 18544 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 25474f26b3ab4143b43529dc11548fe2 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:14:10.728765 18544 raft_consensus.cc:504] T 00000000000000000000000000000000 P 25474f26b3ab4143b43529dc11548fe2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25474f26b3ab4143b43529dc11548fe2" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 46167 } }
I20260807 15:14:10.728895 18544 leader_election.cc:302] T 00000000000000000000000000000000 P 25474f26b3ab4143b43529dc11548fe2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 25474f26b3ab4143b43529dc11548fe2; no voters:
I20260807 15:14:10.729091 18544 leader_election.cc:288] T 00000000000000000000000000000000 P 25474f26b3ab4143b43529dc11548fe2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 15:14:10.729233 18549 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 25474f26b3ab4143b43529dc11548fe2 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:14:10.729410 18544 sys_catalog.cc:564] T 00000000000000000000000000000000 P 25474f26b3ab4143b43529dc11548fe2 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:14:10.729669 18549 raft_consensus.cc:686] T 00000000000000000000000000000000 P 25474f26b3ab4143b43529dc11548fe2 [term 1 LEADER]: Becoming Leader. State: Replica: 25474f26b3ab4143b43529dc11548fe2, State: Running, Role: LEADER
I20260807 15:14:10.729861 18549 consensus_queue.cc:237] T 00000000000000000000000000000000 P 25474f26b3ab4143b43529dc11548fe2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25474f26b3ab4143b43529dc11548fe2" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 46167 } }
Aug 07 15:14:10 dist-test-slave-42z9 krb5kdc[18425](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115650, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.194.62@KRBTEST.COM
I20260807 15:14:10.730558 18552 sys_catalog.cc:455] T 00000000000000000000000000000000 P 25474f26b3ab4143b43529dc11548fe2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 25474f26b3ab4143b43529dc11548fe2. Latest consensus state: current_term: 1 leader_uuid: "25474f26b3ab4143b43529dc11548fe2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25474f26b3ab4143b43529dc11548fe2" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 46167 } } }
I20260807 15:14:10.730670 18552 sys_catalog.cc:458] T 00000000000000000000000000000000 P 25474f26b3ab4143b43529dc11548fe2 [sys.catalog]: This master's current role is: LEADER
I20260807 15:14:10.731151 18551 sys_catalog.cc:455] T 00000000000000000000000000000000 P 25474f26b3ab4143b43529dc11548fe2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "25474f26b3ab4143b43529dc11548fe2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25474f26b3ab4143b43529dc11548fe2" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 46167 } } }
I20260807 15:14:10.731593 18551 sys_catalog.cc:458] T 00000000000000000000000000000000 P 25474f26b3ab4143b43529dc11548fe2 [sys.catalog]: This master's current role is: LEADER
W20260807 15:14:10.731791 18575 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 25474f26b3ab4143b43529dc11548fe2: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 15:14:10.731841 18575 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 15:14:10.731909 18573 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:14:10.732071 18573 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:14:10.733814 18573 catalog_manager.cc:1347] Generated new cluster ID: 52b11fd4412f4b2f82cffd1af5601a58
I20260807 15:14:10.733871 18573 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:14:10.735106 18547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:10.722007 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37506 (local address 127.29.194.62:46167)
0807 15:14:10.722201 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0807 15:14:10.722209 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 15:14:10.722235 (+ 26us) server_negotiation.cc:407] Connection header received
0807 15:14:10.722714 (+ 479us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:14:10.722720 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:14:10.722766 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:14:10.722884 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:14:10.723803 (+ 919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:10.725185 (+ 1382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:10.726386 (+ 1201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:10.727031 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:10.727164 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:10.730934 (+ 3770us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:14:10.730940 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:14:10.730940 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:14:10.730956 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:14:10.732994 (+ 2038us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:14:10.733432 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:14:10.733436 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:14:10.733436 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:14:10.733567 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:14:10.733777 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:14:10.733778 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:14:10.733778 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:14:10.734226 (+ 448us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:14:10.734443 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0807 15:14:10.734677 (+ 234us) server_negotiation.cc:299] Negotiation successful
0807 15:14:10.734876 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":148,"thread_start_us":98,"threads_started":1}
I20260807 15:14:10.750666 18573 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:14:10.751412 18573 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:14:10.754853 18573 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 25474f26b3ab4143b43529dc11548fe2: Generated new TSK 0
I20260807 15:14:10.755077 18573 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.29.194.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.194.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.194.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.1.keytab.
Entry for principal kudu/127.29.194.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.1.keytab.
I20260807 15:14:10.790097 30472 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7fZSIL/build/release/bin/kudu
/tmp/dist-test-task7fZSIL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115480851507-30472-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115480851507-30472-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115480851507-30472-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115480851507-30472-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.194.1:0
--local_ip_for_outbound_sockets=127.29.194.1
--webserver_interface=127.29.194.1
--webserver_port=0
--tserver_master_addrs=127.29.194.62:46167
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.1.keytab
--principal=kudu/127.29.194.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.194.20:43617
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115480851507-30472-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115480851507-30472-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115480851507-30472-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:14:10.881671 18591 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:14:10.881927 18591 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:14:10.881963 18591 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:14:10.883240 18591 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:14:10.883298 18591 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:14:10.883339 18591 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.194.1
I20260807 15:14:10.884994 18591 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.194.20:43617
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115480851507-30472-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115480851507-30472-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.1.keytab
--principal=kudu/127.29.194.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.194.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115480851507-30472-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.194.1
--webserver_port=0
--tserver_master_addrs=127.29.194.62:46167
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18591
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.194.1
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115480851507-30472-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:14:10.885258 18591 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:14:10.885540 18591 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:14:10.891969 18604 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:14:10.891961 18600 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:14:10.891976 18596 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:14:10.892010 18591 server_base.cc:1034] running on GCE node
I20260807 15:14:10.892589 18591 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:14:10.892922 18591 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:14:10.894124 18591 hybrid_clock.cc:648] HybridClock initialized: now 1786115650894098 us; error 44 us; skew 500 ppm
Aug 07 15:14:10 dist-test-slave-42z9 krb5kdc[18425](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115650, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.194.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:14:10.896895 18591 init.cc:377] Logged in from keytab as kudu/127.29.194.1@KRBTEST.COM (short username kudu)
I20260807 15:14:10.897382 18591 webserver.cc:460] Webserver started at http://127.29.194.1:43977/ using document root <none> and password file <none>
I20260807 15:14:10.897643 18591 fs_manager.cc:359] Metadata directory not provided
I20260807 15:14:10.897701 18591 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:14:10.897819 18591 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:14:10.898962 18591 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115480851507-30472-0/minicluster-data/ts-0/data/instance:
uuid: "8dc40eed26fb4a0982ceb06e101d137e"
format_stamp: "Formatted at 2026-08-07 15:14:10 on dist-test-slave-42z9"
server_key: "1df099ade868588b8d17cd7196feb163"
server_key_iv: "04d1e05f9c728096baca1337f00dfa74"
server_key_version: "encryptionkey@0"
I20260807 15:14:10.899402 18591 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115480851507-30472-0/minicluster-data/ts-0/wal/instance:
uuid: "8dc40eed26fb4a0982ceb06e101d137e"
format_stamp: "Formatted at 2026-08-07 15:14:10 on dist-test-slave-42z9"
server_key: "1df099ade868588b8d17cd7196feb163"
server_key_iv: "04d1e05f9c728096baca1337f00dfa74"
server_key_version: "encryptionkey@0"
I20260807 15:14:10.900971 18591 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 15:14:10.902253 18615 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:14:10.902467 18591 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:14:10.902566 18591 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115480851507-30472-0/minicluster-data/ts-0/data,/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115480851507-30472-0/minicluster-data/ts-0/wal
uuid: "8dc40eed26fb4a0982ceb06e101d137e"
format_stamp: "Formatted at 2026-08-07 15:14:10 on dist-test-slave-42z9"
server_key: "1df099ade868588b8d17cd7196feb163"
server_key_iv: "04d1e05f9c728096baca1337f00dfa74"
server_key_version: "encryptionkey@0"
I20260807 15:14:10.902638 18591 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115480851507-30472-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115480851507-30472-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115480851507-30472-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:14:10.918824 18591 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:14:10.921096 18591 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:14:10.921236 18591 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:14:10.921511 18591 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:14:10.921840 18591 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:14:10.921876 18591 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:14:10.921906 18591 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:14:10.921932 18591 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:14:10.927759 18591 rpc_server.cc:307] RPC server started. Bound to: 127.29.194.1:43899
I20260807 15:14:10.927829 18791 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.194.1:43899 every 8 connection(s)
I20260807 15:14:10.928143 18591 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115480851507-30472-0/minicluster-data/ts-0/data/info.pb
Aug 07 15:14:10 dist-test-slave-42z9 krb5kdc[18425](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115650, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.194.1@KRBTEST.COM for kudu/127.29.194.62@KRBTEST.COM
I20260807 15:14:10.936812 30472 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7fZSIL/build/release/bin/kudu as pid 18591
I20260807 15:14:10.936935 30472 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115480851507-30472-0/minicluster-data/ts-0/wal/instance
I20260807 15:14:10.937112 30472 external_mini_cluster.cc:1428] Setting key 37dab387c24272a1a73de75bbcd49b49
I20260807 15:14:10.939028 18547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:10.928932 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.194.1:48819 (local address 127.29.194.62:46167)
0807 15:14:10.929016 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0807 15:14:10.929019 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:14:10.929195 (+ 176us) server_negotiation.cc:407] Connection header received
0807 15:14:10.929857 (+ 662us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:14:10.929858 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:14:10.929872 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:14:10.929949 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:14:10.931068 (+ 1119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:10.931783 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:10.932918 (+ 1135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:10.933178 (+ 260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:10.933267 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:10.936058 (+ 2791us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:14:10.936060 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:14:10.936060 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:14:10.936069 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:14:10.937624 (+ 1555us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:14:10.938051 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:14:10.938052 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:14:10.938053 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:14:10.938115 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:14:10.938350 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:14:10.938351 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:14:10.938352 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:14:10.938479 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:14:10.938539 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0807 15:14:10.938818 (+ 279us) server_negotiation.cc:299] Negotiation successful
0807 15:14:10.938922 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260807 15:14:10.939189 18796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:10.928879 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:46167 (local address 127.29.194.1:48819)
0807 15:14:10.929067 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0807 15:14:10.929105 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0807 15:14:10.929713 (+ 608us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:14:10.930014 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:14:10.930018 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:14:10.930635 (+ 617us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:14:10.930876 (+ 241us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:14:10.930881 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:14:10.931949 (+ 1068us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:14:10.931950 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:14:10.932790 (+ 840us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:14:10.932792 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:14:10.933316 (+ 524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:14:10.933317 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:14:10.933493 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:10.934021 (+ 528us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:14:10.934030 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:14:10.935905 (+ 1875us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:14:10.937703 (+ 1798us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:14:10.937711 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:14:10.937713 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:14:10.937964 (+ 251us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:14:10.938178 (+ 214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:14:10.938179 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:14:10.938180 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:14:10.938287 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:14:10.938544 (+ 257us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:14:10.938551 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:14:10.938692 (+ 141us) client_negotiation.cc:769] Sending connection context
0807 15:14:10.938781 (+ 89us) client_negotiation.cc:240] Negotiation successful
0807 15:14:10.938959 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":86,"threads_started":1}
I20260807 15:14:10.939721 18792 heartbeater.cc:347] Connected to a master server at 127.29.194.62:46167
I20260807 15:14:10.939848 18792 heartbeater.cc:464] Registering TS with master...
I20260807 15:14:10.940068 18792 heartbeater.cc:511] Master 127.29.194.62:46167 requested a full tablet report, sending...
I20260807 15:14:10.940624 18494 ts_manager.cc:194] Registered new tserver with Master: 8dc40eed26fb4a0982ceb06e101d137e (127.29.194.1:43899)
I20260807 15:14:10.941251 18494 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.194.1@KRBTEST.COM'} at 127.29.194.1:48819
WARNING: no policy specified for kudu/127.29.194.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.194.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.194.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.2.keytab.
Entry for principal kudu/127.29.194.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.2.keytab.
I20260807 15:14:10.965394 30472 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7fZSIL/build/release/bin/kudu
/tmp/dist-test-task7fZSIL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115480851507-30472-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115480851507-30472-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115480851507-30472-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115480851507-30472-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.194.2:0
--local_ip_for_outbound_sockets=127.29.194.2
--webserver_interface=127.29.194.2
--webserver_port=0
--tserver_master_addrs=127.29.194.62:46167
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.2.keytab
--principal=kudu/127.29.194.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.194.20:43617
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115480851507-30472-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115480851507-30472-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115480851507-30472-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:14:11.049093 18800 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:14:11.049324 18800 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:14:11.049357 18800 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:14:11.050616 18800 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:14:11.050669 18800 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:14:11.050709 18800 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.194.2
I20260807 15:14:11.052304 18800 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.194.20:43617
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115480851507-30472-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115480851507-30472-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.2.keytab
--principal=kudu/127.29.194.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.194.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115480851507-30472-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.194.2
--webserver_port=0
--tserver_master_addrs=127.29.194.62:46167
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18800
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.194.2
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115480851507-30472-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:14:11.052575 18800 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:14:11.052830 18800 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:14:11.058063 18811 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:14:11.058056 18806 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:14:11.058058 18807 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:14:11.058276 18800 server_base.cc:1034] running on GCE node
I20260807 15:14:11.058525 18800 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:14:11.058775 18800 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:14:11.059986 18800 hybrid_clock.cc:648] HybridClock initialized: now 1786115651059948 us; error 73 us; skew 500 ppm
Aug 07 15:14:11 dist-test-slave-42z9 krb5kdc[18425](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115651, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.194.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:14:11.062826 18800 init.cc:377] Logged in from keytab as kudu/127.29.194.2@KRBTEST.COM (short username kudu)
I20260807 15:14:11.063328 18800 webserver.cc:460] Webserver started at http://127.29.194.2:36331/ using document root <none> and password file <none>
I20260807 15:14:11.063565 18800 fs_manager.cc:359] Metadata directory not provided
I20260807 15:14:11.063624 18800 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:14:11.063723 18800 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:14:11.064772 18800 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115480851507-30472-0/minicluster-data/ts-1/data/instance:
uuid: "ff3aa95770c3407f9a7ec35fe5b235c2"
format_stamp: "Formatted at 2026-08-07 15:14:11 on dist-test-slave-42z9"
server_key: "2138424ff93118e26d34ccb996845cb7"
server_key_iv: "7ea05b46921a36dc1c9400216469249f"
server_key_version: "encryptionkey@0"
I20260807 15:14:11.065097 18800 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115480851507-30472-0/minicluster-data/ts-1/wal/instance:
uuid: "ff3aa95770c3407f9a7ec35fe5b235c2"
format_stamp: "Formatted at 2026-08-07 15:14:11 on dist-test-slave-42z9"
server_key: "2138424ff93118e26d34ccb996845cb7"
server_key_iv: "7ea05b46921a36dc1c9400216469249f"
server_key_version: "encryptionkey@0"
I20260807 15:14:11.066747 18800 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260807 15:14:11.068029 18818 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:14:11.068316 18800 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 15:14:11.068452 18800 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115480851507-30472-0/minicluster-data/ts-1/data,/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115480851507-30472-0/minicluster-data/ts-1/wal
uuid: "ff3aa95770c3407f9a7ec35fe5b235c2"
format_stamp: "Formatted at 2026-08-07 15:14:11 on dist-test-slave-42z9"
server_key: "2138424ff93118e26d34ccb996845cb7"
server_key_iv: "7ea05b46921a36dc1c9400216469249f"
server_key_version: "encryptionkey@0"
I20260807 15:14:11.068548 18800 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115480851507-30472-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115480851507-30472-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115480851507-30472-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:14:11.076721 18800 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:14:11.078976 18800 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:14:11.079107 18800 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:14:11.079306 18800 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:14:11.079612 18800 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:14:11.079643 18800 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:14:11.079665 18800 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:14:11.079684 18800 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:14:11.085945 18800 rpc_server.cc:307] RPC server started. Bound to: 127.29.194.2:33663
I20260807 15:14:11.086020 18991 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.194.2:33663 every 8 connection(s)
I20260807 15:14:11.086319 18800 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115480851507-30472-0/minicluster-data/ts-1/data/info.pb
I20260807 15:14:11.091277 30472 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7fZSIL/build/release/bin/kudu as pid 18800
I20260807 15:14:11.091383 30472 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115480851507-30472-0/minicluster-data/ts-1/wal/instance
I20260807 15:14:11.091598 30472 external_mini_cluster.cc:1428] Setting key 0b126865d31b32c8471ee693bcae769d
Aug 07 15:14:11 dist-test-slave-42z9 krb5kdc[18425](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115651, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.194.2@KRBTEST.COM for kudu/127.29.194.62@KRBTEST.COM
I20260807 15:14:11.096305 18547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:11.087084 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.194.2:60915 (local address 127.29.194.62:46167)
0807 15:14:11.087175 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0807 15:14:11.087177 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:14:11.087397 (+ 220us) server_negotiation.cc:407] Connection header received
0807 15:14:11.087904 (+ 507us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:14:11.087905 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:14:11.087918 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:14:11.087978 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:14:11.088939 (+ 961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:11.089513 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:11.090519 (+ 1006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:11.090725 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:11.090782 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:11.093140 (+ 2358us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:14:11.093142 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:14:11.093142 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:14:11.093150 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:14:11.094720 (+ 1570us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:14:11.095124 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:14:11.095125 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:14:11.095126 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:14:11.095186 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:14:11.095428 (+ 242us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:14:11.095429 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:14:11.095429 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:14:11.095554 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:14:11.095612 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0807 15:14:11.096034 (+ 422us) server_negotiation.cc:299] Negotiation successful
0807 15:14:11.096214 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260807 15:14:11.096323 18994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:11.087036 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:46167 (local address 127.29.194.2:60915)
0807 15:14:11.087242 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0807 15:14:11.087289 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0807 15:14:11.087801 (+ 512us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:14:11.088051 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:14:11.088054 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:14:11.088629 (+ 575us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:14:11.088830 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:14:11.088835 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:14:11.089598 (+ 763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:14:11.089599 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:14:11.090436 (+ 837us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:14:11.090439 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:14:11.090802 (+ 363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:14:11.090803 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:14:11.090944 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:11.091518 (+ 574us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:14:11.091526 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:14:11.093045 (+ 1519us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:14:11.094809 (+ 1764us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:14:11.094818 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:14:11.094820 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:14:11.095049 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:14:11.095246 (+ 197us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:14:11.095247 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:14:11.095248 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:14:11.095363 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:14:11.095618 (+ 255us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:14:11.095626 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:14:11.095794 (+ 168us) client_negotiation.cc:769] Sending connection context
0807 15:14:11.095911 (+ 117us) client_negotiation.cc:240] Negotiation successful
0807 15:14:11.096085 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":104,"threads_started":1}
I20260807 15:14:11.097173 18992 heartbeater.cc:347] Connected to a master server at 127.29.194.62:46167
I20260807 15:14:11.097276 18992 heartbeater.cc:464] Registering TS with master...
I20260807 15:14:11.097620 18992 heartbeater.cc:511] Master 127.29.194.62:46167 requested a full tablet report, sending...
I20260807 15:14:11.098066 18494 ts_manager.cc:194] Registered new tserver with Master: ff3aa95770c3407f9a7ec35fe5b235c2 (127.29.194.2:33663)
I20260807 15:14:11.098449 18494 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.194.2@KRBTEST.COM'} at 127.29.194.2:60915
WARNING: no policy specified for kudu/127.29.194.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.194.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.194.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.3.keytab.
Entry for principal kudu/127.29.194.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.3.keytab.
I20260807 15:14:11.121273 30472 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7fZSIL/build/release/bin/kudu
/tmp/dist-test-task7fZSIL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115480851507-30472-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115480851507-30472-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115480851507-30472-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115480851507-30472-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.194.3:0
--local_ip_for_outbound_sockets=127.29.194.3
--webserver_interface=127.29.194.3
--webserver_port=0
--tserver_master_addrs=127.29.194.62:46167
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.3.keytab
--principal=kudu/127.29.194.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.194.20:43617
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115480851507-30472-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115480851507-30472-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115480851507-30472-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:14:11.214258 18997 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:14:11.214512 18997 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:14:11.214541 18997 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:14:11.215931 18997 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:14:11.215991 18997 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:14:11.216020 18997 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.194.3
I20260807 15:14:11.217689 18997 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.194.20:43617
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115480851507-30472-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115480851507-30472-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.3.keytab
--principal=kudu/127.29.194.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.194.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115480851507-30472-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.194.3
--webserver_port=0
--tserver_master_addrs=127.29.194.62:46167
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18997
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.194.3
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115480851507-30472-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:14:11.217959 18997 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:14:11.218210 18997 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:14:11.223749 19005 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:14:11.223749 19004 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:14:11.223914 19009 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:14:11.224180 18997 server_base.cc:1034] running on GCE node
I20260807 15:14:11.224550 18997 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:14:11.224849 18997 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:14:11.226087 18997 hybrid_clock.cc:648] HybridClock initialized: now 1786115651226053 us; error 37 us; skew 500 ppm
Aug 07 15:14:11 dist-test-slave-42z9 krb5kdc[18425](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115651, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.194.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:14:11.232749 18997 init.cc:377] Logged in from keytab as kudu/127.29.194.3@KRBTEST.COM (short username kudu)
I20260807 15:14:11.233282 18997 webserver.cc:460] Webserver started at http://127.29.194.3:32813/ using document root <none> and password file <none>
I20260807 15:14:11.233575 18997 fs_manager.cc:359] Metadata directory not provided
I20260807 15:14:11.233632 18997 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:14:11.233762 18997 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:14:11.234871 18997 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115480851507-30472-0/minicluster-data/ts-2/data/instance:
uuid: "887081dc41e341b7911a7061f5f6d3a1"
format_stamp: "Formatted at 2026-08-07 15:14:11 on dist-test-slave-42z9"
server_key: "2447b01858ab937b1461a913272b57a5"
server_key_iv: "b75c3f8a4ad4573456a01f06f0fa49a4"
server_key_version: "encryptionkey@0"
I20260807 15:14:11.235281 18997 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115480851507-30472-0/minicluster-data/ts-2/wal/instance:
uuid: "887081dc41e341b7911a7061f5f6d3a1"
format_stamp: "Formatted at 2026-08-07 15:14:11 on dist-test-slave-42z9"
server_key: "2447b01858ab937b1461a913272b57a5"
server_key_iv: "b75c3f8a4ad4573456a01f06f0fa49a4"
server_key_version: "encryptionkey@0"
I20260807 15:14:11.236692 18997 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 15:14:11.237843 19023 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:14:11.238204 18997 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:14:11.238324 18997 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115480851507-30472-0/minicluster-data/ts-2/data,/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115480851507-30472-0/minicluster-data/ts-2/wal
uuid: "887081dc41e341b7911a7061f5f6d3a1"
format_stamp: "Formatted at 2026-08-07 15:14:11 on dist-test-slave-42z9"
server_key: "2447b01858ab937b1461a913272b57a5"
server_key_iv: "b75c3f8a4ad4573456a01f06f0fa49a4"
server_key_version: "encryptionkey@0"
I20260807 15:14:11.238413 18997 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115480851507-30472-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115480851507-30472-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115480851507-30472-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:14:11.254624 18997 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:14:11.257401 18997 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:14:11.257603 18997 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:14:11.257853 18997 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:14:11.258213 18997 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:14:11.258248 18997 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:14:11.258279 18997 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:14:11.258297 18997 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:14:11.265126 18997 rpc_server.cc:307] RPC server started. Bound to: 127.29.194.3:34127
I20260807 15:14:11.265266 19202 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.194.3:34127 every 8 connection(s)
I20260807 15:14:11.265580 18997 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115480851507-30472-0/minicluster-data/ts-2/data/info.pb
I20260807 15:14:11.268757 30472 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7fZSIL/build/release/bin/kudu as pid 18997
I20260807 15:14:11.268863 30472 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115480851507-30472-0/minicluster-data/ts-2/wal/instance
I20260807 15:14:11.269021 30472 external_mini_cluster.cc:1428] Setting key 0e6d9a327281b9513e4b83390d017d8f
Aug 07 15:14:11 dist-test-slave-42z9 krb5kdc[18425](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115651, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.194.3@KRBTEST.COM for kudu/127.29.194.62@KRBTEST.COM
I20260807 15:14:11.276547 18547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:11.266357 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.194.3:47537 (local address 127.29.194.62:46167)
0807 15:14:11.266425 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0807 15:14:11.266427 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:14:11.266631 (+ 204us) server_negotiation.cc:407] Connection header received
0807 15:14:11.267143 (+ 512us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:14:11.267144 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:14:11.267154 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:14:11.267204 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:14:11.268183 (+ 979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:11.268780 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:11.270243 (+ 1463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:11.270447 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:11.270513 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:11.273143 (+ 2630us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:14:11.273144 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:14:11.273145 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:14:11.273154 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:14:11.274684 (+ 1530us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:14:11.275192 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:14:11.275193 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:14:11.275193 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:14:11.275249 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:14:11.275700 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:14:11.275701 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:14:11.275701 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:14:11.275840 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:14:11.275911 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0807 15:14:11.276254 (+ 343us) server_negotiation.cc:299] Negotiation successful
0807 15:14:11.276454 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260807 15:14:11.276593 19206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:11.266325 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:46167 (local address 127.29.194.3:47537)
0807 15:14:11.266508 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0807 15:14:11.266547 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0807 15:14:11.267058 (+ 511us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:14:11.267268 (+ 210us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:14:11.267271 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:14:11.267878 (+ 607us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:14:11.268096 (+ 218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:14:11.268101 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:14:11.268965 (+ 864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:14:11.268967 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:14:11.270142 (+ 1175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:14:11.270145 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:14:11.270525 (+ 380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:14:11.270526 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:14:11.270658 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:11.271154 (+ 496us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:14:11.271162 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:14:11.272967 (+ 1805us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:14:11.274816 (+ 1849us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:14:11.274828 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:14:11.274830 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:14:11.275112 (+ 282us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:14:11.275371 (+ 259us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:14:11.275373 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:14:11.275373 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:14:11.275590 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:14:11.275954 (+ 364us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:14:11.275962 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:14:11.276119 (+ 157us) client_negotiation.cc:769] Sending connection context
0807 15:14:11.276232 (+ 113us) client_negotiation.cc:240] Negotiation successful
0807 15:14:11.276394 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":92,"threads_started":1}
I20260807 15:14:11.277204 19203 heartbeater.cc:347] Connected to a master server at 127.29.194.62:46167
I20260807 15:14:11.277304 19203 heartbeater.cc:464] Registering TS with master...
I20260807 15:14:11.277518 19203 heartbeater.cc:511] Master 127.29.194.62:46167 requested a full tablet report, sending...
I20260807 15:14:11.277865 18494 ts_manager.cc:194] Registered new tserver with Master: 887081dc41e341b7911a7061f5f6d3a1 (127.29.194.3:34127)
I20260807 15:14:11.278234 18494 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.194.3@KRBTEST.COM'} at 127.29.194.3:47537
I20260807 15:14:11.281333 30472 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 15:14:11.290195 18547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:11.283107 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37518 (local address 127.29.194.62:46167)
0807 15:14:11.283211 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0807 15:14:11.283214 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:14:11.283376 (+ 162us) server_negotiation.cc:407] Connection header received
0807 15:14:11.283455 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:14:11.283456 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:14:11.283468 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:14:11.283516 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:14:11.284163 (+ 647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:11.284716 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:11.285509 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:11.285805 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:11.285890 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:11.287313 (+ 1423us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:14:11.287317 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:14:11.287317 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:14:11.287324 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:14:11.288857 (+ 1533us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:14:11.289250 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:14:11.289251 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:14:11.289252 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:14:11.289319 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:14:11.289534 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:14:11.289535 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:14:11.289535 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:14:11.289748 (+ 213us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:14:11.289863 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0807 15:14:11.289989 (+ 126us) server_negotiation.cc:299] Negotiation successful
0807 15:14:11.290083 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260807 15:14:11.292680 18494 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37518:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 15:14:11.294412 18494 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 15:14:11.302474 19229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:11.298278 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40342 (local address 127.29.194.3:34127)
0807 15:14:11.298568 (+ 290us) server_negotiation.cc:206] Beginning negotiation
0807 15:14:11.298574 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 15:14:11.298992 (+ 418us) server_negotiation.cc:407] Connection header received
0807 15:14:11.299083 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:14:11.299085 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:14:11.299102 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:14:11.299176 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:14:11.299306 (+ 130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:11.300424 (+ 1118us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:11.301537 (+ 1113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:11.302051 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:11.302116 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:11.302181 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0807 15:14:11.302323 (+ 142us) server_negotiation.cc:299] Negotiation successful
0807 15:14:11.302366 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":253,"thread_start_us":122,"threads_started":1}
I20260807 15:14:11.302461 19227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:11.298004 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.2:33663 (local address 127.0.0.1:59564)
0807 15:14:11.298284 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0807 15:14:11.298305 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 15:14:11.298391 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:14:11.299011 (+ 620us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:14:11.299011 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:14:11.299014 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:14:11.299072 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:14:11.299074 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:14:11.300034 (+ 960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:14:11.300035 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:14:11.301235 (+ 1200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:14:11.301238 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:14:11.302200 (+ 962us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:14:11.302201 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:14:11.302273 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:11.302279 (+ 6us) client_negotiation.cc:769] Sending connection context
0807 15:14:11.302311 (+ 32us) client_negotiation.cc:240] Negotiation successful
0807 15:14:11.302381 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":177,"mutex_wait_us":25,"thread_start_us":131,"threads_started":1}
I20260807 15:14:11.302587 19232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:11.298645 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59564 (local address 127.29.194.2:33663)
0807 15:14:11.298811 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0807 15:14:11.298816 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:14:11.298828 (+ 12us) server_negotiation.cc:407] Connection header received
0807 15:14:11.298869 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:14:11.298872 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:14:11.298897 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:14:11.299128 (+ 231us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:14:11.299190 (+ 62us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:11.299942 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:11.301333 (+ 1391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:11.302094 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:11.302358 (+ 264us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:11.302420 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0807 15:14:11.302448 (+ 28us) server_negotiation.cc:299] Negotiation successful
0807 15:14:11.302496 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":133,"thread_start_us":82,"threads_started":1}
I20260807 15:14:11.302757 19235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:11.298004 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.3:34127 (local address 127.0.0.1:40342)
0807 15:14:11.298928 (+ 924us) negotiation.cc:107] Waiting for socket to connect
0807 15:14:11.298938 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 15:14:11.298992 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:14:11.299191 (+ 199us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:14:11.299192 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:14:11.299194 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:14:11.299230 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:14:11.299231 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:14:11.300516 (+ 1285us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:14:11.300518 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:14:11.301423 (+ 905us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:14:11.301426 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:14:11.302136 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:14:11.302137 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:14:11.302219 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:11.302234 (+ 15us) client_negotiation.cc:769] Sending connection context
0807 15:14:11.302300 (+ 66us) client_negotiation.cc:240] Negotiation successful
0807 15:14:11.302696 (+ 396us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":905,"mutex_wait_us":772,"thread_start_us":87,"threads_started":1}
I20260807 15:14:11.303326 18891 tablet_service.cc:1467] Processing CreateTablet for tablet 44ab56248a404a3a9412aa8f1d056e78 (DEFAULT_TABLE table=test-table [id=cab4e6f902554161ac8dd5d1920b80b6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:14:11.303433 19101 tablet_service.cc:1467] Processing CreateTablet for tablet 44ab56248a404a3a9412aa8f1d056e78 (DEFAULT_TABLE table=test-table [id=cab4e6f902554161ac8dd5d1920b80b6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:14:11.303642 18891 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 44ab56248a404a3a9412aa8f1d056e78. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:14:11.303788 19101 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 44ab56248a404a3a9412aa8f1d056e78. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:14:11.305701 19228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:11.298003 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.1:43899 (local address 127.0.0.1:50446)
0807 15:14:11.298229 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0807 15:14:11.298284 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0807 15:14:11.298370 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:14:11.298867 (+ 497us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:14:11.298871 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:14:11.298873 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:14:11.298966 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:14:11.298969 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:14:11.300479 (+ 1510us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:14:11.300480 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:14:11.301530 (+ 1050us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:14:11.301534 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:14:11.302389 (+ 855us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:14:11.302390 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:14:11.302449 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:11.302453 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 15:14:11.302479 (+ 26us) client_negotiation.cc:240] Negotiation successful
0807 15:14:11.305572 (+ 3093us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":174,"mutex_wait_us":24,"thread_start_us":72,"threads_started":1}
I20260807 15:14:11.306088 19231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:11.298417 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50446 (local address 127.29.194.1:43899)
0807 15:14:11.298672 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0807 15:14:11.298676 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:14:11.298687 (+ 11us) server_negotiation.cc:407] Connection header received
0807 15:14:11.298737 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:14:11.298740 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:14:11.298756 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:14:11.299052 (+ 296us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:14:11.299127 (+ 75us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:11.300360 (+ 1233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:11.301613 (+ 1253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:11.302300 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:11.305828 (+ 3528us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:11.305923 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0807 15:14:11.305956 (+ 33us) server_negotiation.cc:299] Negotiation successful
0807 15:14:11.306013 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":215,"thread_start_us":99,"threads_started":1}
I20260807 15:14:11.306743 19237 tablet_bootstrap.cc:492] T 44ab56248a404a3a9412aa8f1d056e78 P 887081dc41e341b7911a7061f5f6d3a1: Bootstrap starting.
I20260807 15:14:11.306994 18688 tablet_service.cc:1467] Processing CreateTablet for tablet 44ab56248a404a3a9412aa8f1d056e78 (DEFAULT_TABLE table=test-table [id=cab4e6f902554161ac8dd5d1920b80b6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:14:11.307297 18688 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 44ab56248a404a3a9412aa8f1d056e78. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:14:11.307337 19237 tablet_bootstrap.cc:654] T 44ab56248a404a3a9412aa8f1d056e78 P 887081dc41e341b7911a7061f5f6d3a1: Neither blocks nor log segments found. Creating new log.
I20260807 15:14:11.307782 19237 log.cc:824] T 44ab56248a404a3a9412aa8f1d056e78 P 887081dc41e341b7911a7061f5f6d3a1: Log is configured to *not* fsync() on all Append() calls
I20260807 15:14:11.308849 19237 tablet_bootstrap.cc:492] T 44ab56248a404a3a9412aa8f1d056e78 P 887081dc41e341b7911a7061f5f6d3a1: No bootstrap required, opened a new log
I20260807 15:14:11.308924 19237 ts_tablet_manager.cc:1397] T 44ab56248a404a3a9412aa8f1d056e78 P 887081dc41e341b7911a7061f5f6d3a1: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 15:14:11.310111 19239 tablet_bootstrap.cc:492] T 44ab56248a404a3a9412aa8f1d056e78 P 8dc40eed26fb4a0982ceb06e101d137e: Bootstrap starting.
I20260807 15:14:11.310174 19236 tablet_bootstrap.cc:492] T 44ab56248a404a3a9412aa8f1d056e78 P ff3aa95770c3407f9a7ec35fe5b235c2: Bootstrap starting.
I20260807 15:14:11.310807 19239 tablet_bootstrap.cc:654] T 44ab56248a404a3a9412aa8f1d056e78 P 8dc40eed26fb4a0982ceb06e101d137e: Neither blocks nor log segments found. Creating new log.
I20260807 15:14:11.310807 19236 tablet_bootstrap.cc:654] T 44ab56248a404a3a9412aa8f1d056e78 P ff3aa95770c3407f9a7ec35fe5b235c2: Neither blocks nor log segments found. Creating new log.
I20260807 15:14:11.311040 19237 raft_consensus.cc:348] T 44ab56248a404a3a9412aa8f1d056e78 P 887081dc41e341b7911a7061f5f6d3a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff3aa95770c3407f9a7ec35fe5b235c2" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 33663 } } peers { permanent_uuid: "8dc40eed26fb4a0982ceb06e101d137e" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 43899 } } peers { permanent_uuid: "887081dc41e341b7911a7061f5f6d3a1" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 34127 } }
I20260807 15:14:11.311182 19237 raft_consensus.cc:374] T 44ab56248a404a3a9412aa8f1d056e78 P 887081dc41e341b7911a7061f5f6d3a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:14:11.311215 19237 raft_consensus.cc:729] T 44ab56248a404a3a9412aa8f1d056e78 P 887081dc41e341b7911a7061f5f6d3a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 887081dc41e341b7911a7061f5f6d3a1, State: Initialized, Role: FOLLOWER
I20260807 15:14:11.311338 19237 consensus_queue.cc:260] T 44ab56248a404a3a9412aa8f1d056e78 P 887081dc41e341b7911a7061f5f6d3a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff3aa95770c3407f9a7ec35fe5b235c2" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 33663 } } peers { permanent_uuid: "8dc40eed26fb4a0982ceb06e101d137e" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 43899 } } peers { permanent_uuid: "887081dc41e341b7911a7061f5f6d3a1" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 34127 } }
I20260807 15:14:11.311371 19239 log.cc:824] T 44ab56248a404a3a9412aa8f1d056e78 P 8dc40eed26fb4a0982ceb06e101d137e: Log is configured to *not* fsync() on all Append() calls
I20260807 15:14:11.311414 19236 log.cc:824] T 44ab56248a404a3a9412aa8f1d056e78 P ff3aa95770c3407f9a7ec35fe5b235c2: Log is configured to *not* fsync() on all Append() calls
I20260807 15:14:11.311630 19237 ts_tablet_manager.cc:1428] T 44ab56248a404a3a9412aa8f1d056e78 P 887081dc41e341b7911a7061f5f6d3a1: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260807 15:14:11.311731 19203 heartbeater.cc:503] Master 127.29.194.62:46167 was elected leader, sending a full tablet report...
I20260807 15:14:11.312202 19236 tablet_bootstrap.cc:492] T 44ab56248a404a3a9412aa8f1d056e78 P ff3aa95770c3407f9a7ec35fe5b235c2: No bootstrap required, opened a new log
I20260807 15:14:11.312266 19236 ts_tablet_manager.cc:1397] T 44ab56248a404a3a9412aa8f1d056e78 P ff3aa95770c3407f9a7ec35fe5b235c2: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20260807 15:14:11.312381 19239 tablet_bootstrap.cc:492] T 44ab56248a404a3a9412aa8f1d056e78 P 8dc40eed26fb4a0982ceb06e101d137e: No bootstrap required, opened a new log
I20260807 15:14:11.312469 19239 ts_tablet_manager.cc:1397] T 44ab56248a404a3a9412aa8f1d056e78 P 8dc40eed26fb4a0982ceb06e101d137e: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260807 15:14:11.313841 19236 raft_consensus.cc:348] T 44ab56248a404a3a9412aa8f1d056e78 P ff3aa95770c3407f9a7ec35fe5b235c2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff3aa95770c3407f9a7ec35fe5b235c2" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 33663 } } peers { permanent_uuid: "8dc40eed26fb4a0982ceb06e101d137e" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 43899 } } peers { permanent_uuid: "887081dc41e341b7911a7061f5f6d3a1" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 34127 } }
I20260807 15:14:11.314019 19236 raft_consensus.cc:374] T 44ab56248a404a3a9412aa8f1d056e78 P ff3aa95770c3407f9a7ec35fe5b235c2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:14:11.314060 19236 raft_consensus.cc:729] T 44ab56248a404a3a9412aa8f1d056e78 P ff3aa95770c3407f9a7ec35fe5b235c2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ff3aa95770c3407f9a7ec35fe5b235c2, State: Initialized, Role: FOLLOWER
I20260807 15:14:11.313946 19239 raft_consensus.cc:348] T 44ab56248a404a3a9412aa8f1d056e78 P 8dc40eed26fb4a0982ceb06e101d137e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff3aa95770c3407f9a7ec35fe5b235c2" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 33663 } } peers { permanent_uuid: "8dc40eed26fb4a0982ceb06e101d137e" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 43899 } } peers { permanent_uuid: "887081dc41e341b7911a7061f5f6d3a1" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 34127 } }
I20260807 15:14:11.314118 19239 raft_consensus.cc:374] T 44ab56248a404a3a9412aa8f1d056e78 P 8dc40eed26fb4a0982ceb06e101d137e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:14:11.314152 19239 raft_consensus.cc:729] T 44ab56248a404a3a9412aa8f1d056e78 P 8dc40eed26fb4a0982ceb06e101d137e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8dc40eed26fb4a0982ceb06e101d137e, State: Initialized, Role: FOLLOWER
I20260807 15:14:11.314174 19236 consensus_queue.cc:260] T 44ab56248a404a3a9412aa8f1d056e78 P ff3aa95770c3407f9a7ec35fe5b235c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff3aa95770c3407f9a7ec35fe5b235c2" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 33663 } } peers { permanent_uuid: "8dc40eed26fb4a0982ceb06e101d137e" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 43899 } } peers { permanent_uuid: "887081dc41e341b7911a7061f5f6d3a1" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 34127 } }
I20260807 15:14:11.314258 19239 consensus_queue.cc:260] T 44ab56248a404a3a9412aa8f1d056e78 P 8dc40eed26fb4a0982ceb06e101d137e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff3aa95770c3407f9a7ec35fe5b235c2" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 33663 } } peers { permanent_uuid: "8dc40eed26fb4a0982ceb06e101d137e" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 43899 } } peers { permanent_uuid: "887081dc41e341b7911a7061f5f6d3a1" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 34127 } }
I20260807 15:14:11.314510 18992 heartbeater.cc:503] Master 127.29.194.62:46167 was elected leader, sending a full tablet report...
I20260807 15:14:11.314553 19239 ts_tablet_manager.cc:1428] T 44ab56248a404a3a9412aa8f1d056e78 P 8dc40eed26fb4a0982ceb06e101d137e: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260807 15:14:11.314563 18792 heartbeater.cc:503] Master 127.29.194.62:46167 was elected leader, sending a full tablet report...
I20260807 15:14:11.314484 19236 ts_tablet_manager.cc:1428] T 44ab56248a404a3a9412aa8f1d056e78 P ff3aa95770c3407f9a7ec35fe5b235c2: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
W20260807 15:14:11.336889 18993 tablet.cc:2369] T 44ab56248a404a3a9412aa8f1d056e78 P ff3aa95770c3407f9a7ec35fe5b235c2: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 15:14:11.429137 18794 tablet.cc:2369] T 44ab56248a404a3a9412aa8f1d056e78 P 8dc40eed26fb4a0982ceb06e101d137e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 15:14:11.514240 19241 raft_consensus.cc:482] T 44ab56248a404a3a9412aa8f1d056e78 P 887081dc41e341b7911a7061f5f6d3a1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 15:14:11.514384 19241 raft_consensus.cc:504] T 44ab56248a404a3a9412aa8f1d056e78 P 887081dc41e341b7911a7061f5f6d3a1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff3aa95770c3407f9a7ec35fe5b235c2" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 33663 } } peers { permanent_uuid: "8dc40eed26fb4a0982ceb06e101d137e" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 43899 } } peers { permanent_uuid: "887081dc41e341b7911a7061f5f6d3a1" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 34127 } }
I20260807 15:14:11.514796 19241 leader_election.cc:288] T 44ab56248a404a3a9412aa8f1d056e78 P 887081dc41e341b7911a7061f5f6d3a1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ff3aa95770c3407f9a7ec35fe5b235c2 (127.29.194.2:33663), 8dc40eed26fb4a0982ceb06e101d137e (127.29.194.1:43899)
W20260807 15:14:11.516124 19205 tablet.cc:2369] T 44ab56248a404a3a9412aa8f1d056e78 P 887081dc41e341b7911a7061f5f6d3a1: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 15:14:11.518405 19206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:11.515165 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.1:43899 (local address 127.29.194.3:38423)
0807 15:14:11.515260 (+ 95us) negotiation.cc:107] Waiting for socket to connect
0807 15:14:11.515274 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 15:14:11.515336 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:14:11.515631 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:14:11.515633 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:14:11.515637 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:14:11.515743 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:14:11.515746 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:14:11.516603 (+ 857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:14:11.516605 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:14:11.517426 (+ 821us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:14:11.517429 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:14:11.518129 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:14:11.518130 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:14:11.518214 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:11.518218 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 15:14:11.518288 (+ 70us) client_negotiation.cc:240] Negotiation successful
0807 15:14:11.518329 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":67}
I20260807 15:14:11.518414 19231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:11.515258 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.194.3:38423 (local address 127.29.194.1:43899)
0807 15:14:11.515307 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0807 15:14:11.515309 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:14:11.515323 (+ 14us) server_negotiation.cc:407] Connection header received
0807 15:14:11.515421 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:14:11.515423 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:14:11.515435 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:14:11.515545 (+ 110us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:14:11.515845 (+ 300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:11.516517 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:11.517532 (+ 1015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:11.518048 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:11.518101 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:11.518109 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0807 15:14:11.518321 (+ 212us) server_negotiation.cc:299] Negotiation successful
0807 15:14:11.518363 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260807 15:14:11.518777 19247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:11.515165 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.2:33663 (local address 127.29.194.3:38103)
0807 15:14:11.515495 (+ 330us) negotiation.cc:107] Waiting for socket to connect
0807 15:14:11.515510 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 15:14:11.515584 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:14:11.515840 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:14:11.515841 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:14:11.515844 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:14:11.515913 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:14:11.515915 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:14:11.516908 (+ 993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:14:11.516910 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:14:11.517849 (+ 939us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:14:11.517851 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:14:11.518544 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:14:11.518545 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:14:11.518618 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:11.518622 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 15:14:11.518671 (+ 49us) client_negotiation.cc:240] Negotiation successful
0807 15:14:11.518709 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":302,"mutex_wait_us":37,"thread_start_us":180,"threads_started":1}
I20260807 15:14:11.518793 19232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:11.515198 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.194.3:38103 (local address 127.29.194.2:33663)
0807 15:14:11.515252 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0807 15:14:11.515254 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:14:11.515623 (+ 369us) server_negotiation.cc:407] Connection header received
0807 15:14:11.515653 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:14:11.515654 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:14:11.515666 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:14:11.515765 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:14:11.516069 (+ 304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:11.516809 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:11.517940 (+ 1131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:11.518441 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:11.518504 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:11.518512 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0807 15:14:11.518697 (+ 185us) server_negotiation.cc:299] Negotiation successful
0807 15:14:11.518744 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260807 15:14:11.518800 18719 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "44ab56248a404a3a9412aa8f1d056e78" candidate_uuid: "887081dc41e341b7911a7061f5f6d3a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8dc40eed26fb4a0982ceb06e101d137e" is_pre_election: true
I20260807 15:14:11.518975 18719 raft_consensus.cc:2454] T 44ab56248a404a3a9412aa8f1d056e78 P 8dc40eed26fb4a0982ceb06e101d137e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 887081dc41e341b7911a7061f5f6d3a1 in term 0.
I20260807 15:14:11.519080 18922 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "44ab56248a404a3a9412aa8f1d056e78" candidate_uuid: "887081dc41e341b7911a7061f5f6d3a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff3aa95770c3407f9a7ec35fe5b235c2" is_pre_election: true
I20260807 15:14:11.519217 19026 leader_election.cc:302] T 44ab56248a404a3a9412aa8f1d056e78 P 887081dc41e341b7911a7061f5f6d3a1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 887081dc41e341b7911a7061f5f6d3a1, 8dc40eed26fb4a0982ceb06e101d137e; no voters:
I20260807 15:14:11.519237 18922 raft_consensus.cc:2454] T 44ab56248a404a3a9412aa8f1d056e78 P ff3aa95770c3407f9a7ec35fe5b235c2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 887081dc41e341b7911a7061f5f6d3a1 in term 0.
I20260807 15:14:11.519382 19241 raft_consensus.cc:2781] T 44ab56248a404a3a9412aa8f1d056e78 P 887081dc41e341b7911a7061f5f6d3a1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 15:14:11.519438 19241 raft_consensus.cc:482] T 44ab56248a404a3a9412aa8f1d056e78 P 887081dc41e341b7911a7061f5f6d3a1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 15:14:11.519464 19241 raft_consensus.cc:3037] T 44ab56248a404a3a9412aa8f1d056e78 P 887081dc41e341b7911a7061f5f6d3a1 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:14:11.520162 19241 raft_consensus.cc:504] T 44ab56248a404a3a9412aa8f1d056e78 P 887081dc41e341b7911a7061f5f6d3a1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff3aa95770c3407f9a7ec35fe5b235c2" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 33663 } } peers { permanent_uuid: "8dc40eed26fb4a0982ceb06e101d137e" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 43899 } } peers { permanent_uuid: "887081dc41e341b7911a7061f5f6d3a1" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 34127 } }
I20260807 15:14:11.520303 19241 leader_election.cc:288] T 44ab56248a404a3a9412aa8f1d056e78 P 887081dc41e341b7911a7061f5f6d3a1 [CANDIDATE]: Term 1 election: Requested vote from peers ff3aa95770c3407f9a7ec35fe5b235c2 (127.29.194.2:33663), 8dc40eed26fb4a0982ceb06e101d137e (127.29.194.1:43899)
I20260807 15:14:11.520491 18719 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "44ab56248a404a3a9412aa8f1d056e78" candidate_uuid: "887081dc41e341b7911a7061f5f6d3a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8dc40eed26fb4a0982ceb06e101d137e"
I20260807 15:14:11.520569 18719 raft_consensus.cc:3037] T 44ab56248a404a3a9412aa8f1d056e78 P 8dc40eed26fb4a0982ceb06e101d137e [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:14:11.520627 18922 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "44ab56248a404a3a9412aa8f1d056e78" candidate_uuid: "887081dc41e341b7911a7061f5f6d3a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff3aa95770c3407f9a7ec35fe5b235c2"
I20260807 15:14:11.520705 18922 raft_consensus.cc:3037] T 44ab56248a404a3a9412aa8f1d056e78 P ff3aa95770c3407f9a7ec35fe5b235c2 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:14:11.521503 18719 raft_consensus.cc:2454] T 44ab56248a404a3a9412aa8f1d056e78 P 8dc40eed26fb4a0982ceb06e101d137e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 887081dc41e341b7911a7061f5f6d3a1 in term 1.
I20260807 15:14:11.521567 18922 raft_consensus.cc:2454] T 44ab56248a404a3a9412aa8f1d056e78 P ff3aa95770c3407f9a7ec35fe5b235c2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 887081dc41e341b7911a7061f5f6d3a1 in term 1.
I20260807 15:14:11.521709 19026 leader_election.cc:302] T 44ab56248a404a3a9412aa8f1d056e78 P 887081dc41e341b7911a7061f5f6d3a1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 887081dc41e341b7911a7061f5f6d3a1, 8dc40eed26fb4a0982ceb06e101d137e; no voters:
I20260807 15:14:11.521797 19241 raft_consensus.cc:2781] T 44ab56248a404a3a9412aa8f1d056e78 P 887081dc41e341b7911a7061f5f6d3a1 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:14:11.521963 19241 raft_consensus.cc:686] T 44ab56248a404a3a9412aa8f1d056e78 P 887081dc41e341b7911a7061f5f6d3a1 [term 1 LEADER]: Becoming Leader. State: Replica: 887081dc41e341b7911a7061f5f6d3a1, State: Running, Role: LEADER
I20260807 15:14:11.522071 19241 consensus_queue.cc:237] T 44ab56248a404a3a9412aa8f1d056e78 P 887081dc41e341b7911a7061f5f6d3a1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff3aa95770c3407f9a7ec35fe5b235c2" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 33663 } } peers { permanent_uuid: "8dc40eed26fb4a0982ceb06e101d137e" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 43899 } } peers { permanent_uuid: "887081dc41e341b7911a7061f5f6d3a1" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 34127 } }
I20260807 15:14:11.522920 18490 catalog_manager.cc:5495] T 44ab56248a404a3a9412aa8f1d056e78 P 887081dc41e341b7911a7061f5f6d3a1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 887081dc41e341b7911a7061f5f6d3a1 (127.29.194.3). New cstate: current_term: 1 leader_uuid: "887081dc41e341b7911a7061f5f6d3a1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff3aa95770c3407f9a7ec35fe5b235c2" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 33663 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8dc40eed26fb4a0982ceb06e101d137e" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 43899 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "887081dc41e341b7911a7061f5f6d3a1" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 34127 } health_report { overall_health: HEALTHY } } }
I20260807 15:14:11.530849 19229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:11.528355 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40356 (local address 127.29.194.3:34127)
0807 15:14:11.528409 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0807 15:14:11.528411 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:14:11.528419 (+ 8us) server_negotiation.cc:407] Connection header received
0807 15:14:11.528453 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:14:11.528454 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:14:11.528470 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:14:11.528526 (+ 56us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 15:14:11.528723 (+ 197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:11.529206 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:11.529978 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:11.530238 (+ 260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:11.530305 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:11.530498 (+ 193us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 15:14:11.530612 (+ 114us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 15:14:11.530684 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0807 15:14:11.530729 (+ 45us) server_negotiation.cc:299] Negotiation successful
0807 15:14:11.530751 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260807 15:14:11.532665 18719 raft_consensus.cc:1261] T 44ab56248a404a3a9412aa8f1d056e78 P 8dc40eed26fb4a0982ceb06e101d137e [term 1 FOLLOWER]: Refusing update from remote peer 887081dc41e341b7911a7061f5f6d3a1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:14:11.532687 18922 raft_consensus.cc:1261] T 44ab56248a404a3a9412aa8f1d056e78 P ff3aa95770c3407f9a7ec35fe5b235c2 [term 1 FOLLOWER]: Refusing update from remote peer 887081dc41e341b7911a7061f5f6d3a1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:14:11.533056 19249 consensus_queue.cc:1035] T 44ab56248a404a3a9412aa8f1d056e78 P 887081dc41e341b7911a7061f5f6d3a1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff3aa95770c3407f9a7ec35fe5b235c2" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 33663 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:14:11.533226 19241 consensus_queue.cc:1035] T 44ab56248a404a3a9412aa8f1d056e78 P 887081dc41e341b7911a7061f5f6d3a1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8dc40eed26fb4a0982ceb06e101d137e" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 43899 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:14:11.536725 19254 mvcc.cc:205] Tried to move back new op lower bound from 7315929708675112960 to 7315929708634923008. Current Snapshot: MvccSnapshot[applied={T|T < 7315929708675112960 or (T in {7315929708675112960})}]
I20260807 15:14:11.536854 19253 mvcc.cc:205] Tried to move back new op lower bound from 7315929708675112960 to 7315929708634923008. Current Snapshot: MvccSnapshot[applied={T|T < 7315929708675112960 or (T in {7315929708675112960})}]
I20260807 15:14:11.537360 19255 mvcc.cc:205] Tried to move back new op lower bound from 7315929708675112960 to 7315929708634923008. Current Snapshot: MvccSnapshot[applied={T|T < 7315929708675112960 or (T in {7315929708675112960})}]
I20260807 15:14:11.537638 18490 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37518:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 15:14:11.537737 18490 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37518:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 15:14:11.538777 18490 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 25474f26b3ab4143b43529dc11548fe2: Sending DeleteTablet for 3 replicas of tablet 44ab56248a404a3a9412aa8f1d056e78
I20260807 15:14:11.539129 18688 tablet_service.cc:1514] Processing DeleteTablet for tablet 44ab56248a404a3a9412aa8f1d056e78 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:14:11 UTC) from {username='kudu', principal='kudu/127.29.194.62@KRBTEST.COM'} at 127.0.0.1:50446
I20260807 15:14:11.539160 30472 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7fZSIL/build/release/bin/kudu with pid 18591
I20260807 15:14:11.539160 18891 tablet_service.cc:1514] Processing DeleteTablet for tablet 44ab56248a404a3a9412aa8f1d056e78 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:14:11 UTC) from {username='kudu', principal='kudu/127.29.194.62@KRBTEST.COM'} at 127.0.0.1:59564
I20260807 15:14:11.539402 19269 tablet_replica.cc:330] T 44ab56248a404a3a9412aa8f1d056e78 P ff3aa95770c3407f9a7ec35fe5b235c2: stopping tablet replica
I20260807 15:14:11.539492 19269 raft_consensus.cc:2229] T 44ab56248a404a3a9412aa8f1d056e78 P ff3aa95770c3407f9a7ec35fe5b235c2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 15:14:11.539583 19269 raft_consensus.cc:2258] T 44ab56248a404a3a9412aa8f1d056e78 P ff3aa95770c3407f9a7ec35fe5b235c2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:14:11.548343 19101 tablet_service.cc:1514] Processing DeleteTablet for tablet 44ab56248a404a3a9412aa8f1d056e78 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:14:11 UTC) from {username='kudu', principal='kudu/127.29.194.62@KRBTEST.COM'} at 127.0.0.1:40342
I20260807 15:14:11.549867 19270 tablet_replica.cc:330] T 44ab56248a404a3a9412aa8f1d056e78 P 887081dc41e341b7911a7061f5f6d3a1: stopping tablet replica
I20260807 15:14:11.549986 19270 raft_consensus.cc:2229] T 44ab56248a404a3a9412aa8f1d056e78 P 887081dc41e341b7911a7061f5f6d3a1 [term 1 LEADER]: Raft consensus shutting down.
I20260807 15:14:11.550087 19270 raft_consensus.cc:2258] T 44ab56248a404a3a9412aa8f1d056e78 P 887081dc41e341b7911a7061f5f6d3a1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:14:11.552258 19270 ts_tablet_manager.cc:1905] T 44ab56248a404a3a9412aa8f1d056e78 P 887081dc41e341b7911a7061f5f6d3a1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:14:11.554253 19270 ts_tablet_manager.cc:1918] T 44ab56248a404a3a9412aa8f1d056e78 P 887081dc41e341b7911a7061f5f6d3a1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 15:14:11.554324 19270 log.cc:1196] T 44ab56248a404a3a9412aa8f1d056e78 P 887081dc41e341b7911a7061f5f6d3a1: Deleting WAL directory at /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115480851507-30472-0/minicluster-data/ts-2/wal/wals/44ab56248a404a3a9412aa8f1d056e78
I20260807 15:14:11.554531 19270 ts_tablet_manager.cc:1939] T 44ab56248a404a3a9412aa8f1d056e78 P 887081dc41e341b7911a7061f5f6d3a1: Deleting consensus metadata
I20260807 15:14:11.554940 18474 catalog_manager.cc:4841] TS 887081dc41e341b7911a7061f5f6d3a1 (127.29.194.3:34127): tablet 44ab56248a404a3a9412aa8f1d056e78 (table test-table [id=cab4e6f902554161ac8dd5d1920b80b6]) successfully deleted
I20260807 15:14:11.557891 19269 ts_tablet_manager.cc:1905] T 44ab56248a404a3a9412aa8f1d056e78 P ff3aa95770c3407f9a7ec35fe5b235c2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:14:11.559324 19269 ts_tablet_manager.cc:1918] T 44ab56248a404a3a9412aa8f1d056e78 P ff3aa95770c3407f9a7ec35fe5b235c2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 15:14:11.559414 19269 log.cc:1196] T 44ab56248a404a3a9412aa8f1d056e78 P ff3aa95770c3407f9a7ec35fe5b235c2: Deleting WAL directory at /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115480851507-30472-0/minicluster-data/ts-1/wal/wals/44ab56248a404a3a9412aa8f1d056e78
I20260807 15:14:11.559661 19269 ts_tablet_manager.cc:1939] T 44ab56248a404a3a9412aa8f1d056e78 P ff3aa95770c3407f9a7ec35fe5b235c2: Deleting consensus metadata
I20260807 15:14:11.560094 18476 catalog_manager.cc:4841] TS ff3aa95770c3407f9a7ec35fe5b235c2 (127.29.194.2:33663): tablet 44ab56248a404a3a9412aa8f1d056e78 (table test-table [id=cab4e6f902554161ac8dd5d1920b80b6]) successfully deleted
W20260807 15:14:11.560257 18473 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.29.194.1:43899 (error 108)
I20260807 15:14:11.560600 30472 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7fZSIL/build/release/bin/kudu with pid 18800
I20260807 15:14:11.560752 19228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:11.560635 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.1:43899 (local address 127.0.0.1:50456)
0807 15:14:11.560683 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0807 15:14:11.560719 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.194.1:43899: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260807 15:14:11.593667 18473 catalog_manager.cc:4584] TS 8dc40eed26fb4a0982ceb06e101d137e (127.29.194.1:43899): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 44ab56248a404a3a9412aa8f1d056e78: Network error: Client connection negotiation failed: client connection to 127.29.194.1:43899: connect: Connection refused (error 111)
I20260807 15:14:11.599576 30472 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7fZSIL/build/release/bin/kudu with pid 18997
I20260807 15:14:11.616469 30472 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7fZSIL/build/release/bin/kudu with pid 18445
I20260807 15:14:11.630568 30472 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7fZSIL/build/release/bin/kudu
/tmp/dist-test-task7fZSIL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115480851507-30472-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115480851507-30472-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115480851507-30472-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115480851507-30472-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.194.62:46167
--webserver_interface=127.29.194.62
--webserver_port=42443
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.62.keytab
--principal=kudu/127.29.194.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.194.20:43617
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.194.62:46167
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115480851507-30472-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115480851507-30472-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115480851507-30472-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:14:11.714326 19274 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:14:11.714671 19274 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:14:11.714697 19274 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:14:11.715958 19274 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:14:11.716008 19274 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:14:11.716022 19274 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:14:11.716032 19274 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:14:11.716051 19274 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:14:11.717617 19274 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.194.20:43617
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115480851507-30472-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115480851507-30472-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.29.194.62:46167
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.62.keytab
--principal=kudu/127.29.194.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.194.62:46167
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115480851507-30472-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.194.62
--webserver_port=42443
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19274
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115480851507-30472-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:14:11.717847 19274 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:14:11.718081 19274 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:14:11.727568 19281 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:14:11.727569 19282 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:14:11.727612 19274 server_base.cc:1034] running on GCE node
W20260807 15:14:11.727574 19284 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:14:11.728127 19274 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:14:11.728428 19274 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:14:11.729583 19274 hybrid_clock.cc:648] HybridClock initialized: now 1786115651729564 us; error 36 us; skew 500 ppm
Aug 07 15:14:11 dist-test-slave-42z9 krb5kdc[18425](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115651, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.194.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:14:11.732215 19274 init.cc:377] Logged in from keytab as kudu/127.29.194.62@KRBTEST.COM (short username kudu)
I20260807 15:14:11.732694 19274 webserver.cc:460] Webserver started at http://127.29.194.62:42443/ using document root <none> and password file <none>
I20260807 15:14:11.732944 19274 fs_manager.cc:359] Metadata directory not provided
I20260807 15:14:11.733003 19274 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:14:11.734436 19274 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:14:11.735220 19293 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:14:11.736176 19274 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:14:11.736263 19274 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115480851507-30472-0/minicluster-data/master-0/data,/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115480851507-30472-0/minicluster-data/master-0/wal
uuid: "25474f26b3ab4143b43529dc11548fe2"
format_stamp: "Formatted at 2026-08-07 15:14:10 on dist-test-slave-42z9"
server_key: "5499c9bcc5ff9f5b2103e1aa0425b153"
server_key_iv: "12cdb2959cfffadfd57e8f3f376805c6"
server_key_version: "encryptionkey@0"
I20260807 15:14:11.736712 19274 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115480851507-30472-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115480851507-30472-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115480851507-30472-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:14:11.757754 19274 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:14:11.760210 19274 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:14:11.760360 19274 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:14:11.764580 19274 rpc_server.cc:307] RPC server started. Bound to: 127.29.194.62:46167
I20260807 15:14:11.764633 19373 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.194.62:46167 every 8 connection(s)
I20260807 15:14:11.765023 19274 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115480851507-30472-0/minicluster-data/master-0/data/info.pb
I20260807 15:14:11.766721 30472 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7fZSIL/build/release/bin/kudu as pid 19274
I20260807 15:14:11.767016 30472 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7fZSIL/build/release/bin/kudu
/tmp/dist-test-task7fZSIL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115480851507-30472-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115480851507-30472-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115480851507-30472-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115480851507-30472-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.194.1:43899
--local_ip_for_outbound_sockets=127.29.194.1
--tserver_master_addrs=127.29.194.62:46167
--webserver_port=43977
--webserver_interface=127.29.194.1
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.1.keytab
--principal=kudu/127.29.194.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.194.20:43617
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115480851507-30472-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115480851507-30472-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115480851507-30472-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 15:14:11.767035 19375 sys_catalog.cc:263] Verifying existing consensus state
I20260807 15:14:11.767740 19375 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 25474f26b3ab4143b43529dc11548fe2: Bootstrap starting.
I20260807 15:14:11.778347 19375 log.cc:824] T 00000000000000000000000000000000 P 25474f26b3ab4143b43529dc11548fe2: Log is configured to *not* fsync() on all Append() calls
I20260807 15:14:11.780380 19375 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 25474f26b3ab4143b43529dc11548fe2: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 15:14:11.780704 19375 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 25474f26b3ab4143b43529dc11548fe2: Bootstrap complete.
I20260807 15:14:11.783262 19375 raft_consensus.cc:348] T 00000000000000000000000000000000 P 25474f26b3ab4143b43529dc11548fe2 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25474f26b3ab4143b43529dc11548fe2" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 46167 } }
I20260807 15:14:11.783696 19375 raft_consensus.cc:729] T 00000000000000000000000000000000 P 25474f26b3ab4143b43529dc11548fe2 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 25474f26b3ab4143b43529dc11548fe2, State: Initialized, Role: FOLLOWER
I20260807 15:14:11.783864 19375 consensus_queue.cc:260] T 00000000000000000000000000000000 P 25474f26b3ab4143b43529dc11548fe2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25474f26b3ab4143b43529dc11548fe2" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 46167 } }
I20260807 15:14:11.783939 19375 raft_consensus.cc:388] T 00000000000000000000000000000000 P 25474f26b3ab4143b43529dc11548fe2 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:14:11.784004 19375 raft_consensus.cc:482] T 00000000000000000000000000000000 P 25474f26b3ab4143b43529dc11548fe2 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:14:11.784054 19375 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 25474f26b3ab4143b43529dc11548fe2 [term 1 FOLLOWER]: Advancing to term 2
I20260807 15:14:11.785118 19375 raft_consensus.cc:504] T 00000000000000000000000000000000 P 25474f26b3ab4143b43529dc11548fe2 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25474f26b3ab4143b43529dc11548fe2" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 46167 } }
I20260807 15:14:11.785221 19375 leader_election.cc:302] T 00000000000000000000000000000000 P 25474f26b3ab4143b43529dc11548fe2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 25474f26b3ab4143b43529dc11548fe2; no voters:
I20260807 15:14:11.785401 19375 leader_election.cc:288] T 00000000000000000000000000000000 P 25474f26b3ab4143b43529dc11548fe2 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260807 15:14:11.785497 19380 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 25474f26b3ab4143b43529dc11548fe2 [term 2 FOLLOWER]: Leader election won for term 2
I20260807 15:14:11.785725 19380 raft_consensus.cc:686] T 00000000000000000000000000000000 P 25474f26b3ab4143b43529dc11548fe2 [term 2 LEADER]: Becoming Leader. State: Replica: 25474f26b3ab4143b43529dc11548fe2, State: Running, Role: LEADER
I20260807 15:14:11.785709 19375 sys_catalog.cc:564] T 00000000000000000000000000000000 P 25474f26b3ab4143b43529dc11548fe2 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:14:11.785905 19380 consensus_queue.cc:237] T 00000000000000000000000000000000 P 25474f26b3ab4143b43529dc11548fe2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25474f26b3ab4143b43529dc11548fe2" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 46167 } }
I20260807 15:14:11.786291 19382 sys_catalog.cc:455] T 00000000000000000000000000000000 P 25474f26b3ab4143b43529dc11548fe2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 25474f26b3ab4143b43529dc11548fe2. Latest consensus state: current_term: 2 leader_uuid: "25474f26b3ab4143b43529dc11548fe2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25474f26b3ab4143b43529dc11548fe2" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 46167 } } }
I20260807 15:14:11.786370 19382 sys_catalog.cc:458] T 00000000000000000000000000000000 P 25474f26b3ab4143b43529dc11548fe2 [sys.catalog]: This master's current role is: LEADER
I20260807 15:14:11.786679 19390 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:14:11.786746 19381 sys_catalog.cc:455] T 00000000000000000000000000000000 P 25474f26b3ab4143b43529dc11548fe2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "25474f26b3ab4143b43529dc11548fe2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25474f26b3ab4143b43529dc11548fe2" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 46167 } } }
I20260807 15:14:11.786804 19381 sys_catalog.cc:458] T 00000000000000000000000000000000 P 25474f26b3ab4143b43529dc11548fe2 [sys.catalog]: This master's current role is: LEADER
I20260807 15:14:11.787482 19390 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:14:11.787573 19390 catalog_manager.cc:1259] Loaded cluster ID: 52b11fd4412f4b2f82cffd1af5601a58
I20260807 15:14:11.787601 19390 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:14:11.787829 19390 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260807 15:14:11.790768 19390 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260807 15:14:11.794898 19390 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:14:11.795167 19390 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 25474f26b3ab4143b43529dc11548fe2: Loaded TSK: 0
I20260807 15:14:11.795373 19390 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260807 15:14:11.867249 19377 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:14:11.867538 19377 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:14:11.867564 19377 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:14:11.868863 19377 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:14:11.868920 19377 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:14:11.868949 19377 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.194.1
I20260807 15:14:11.870620 19377 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.194.20:43617
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115480851507-30472-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115480851507-30472-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.1.keytab
--principal=kudu/127.29.194.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.194.1:43899
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115480851507-30472-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.194.1
--webserver_port=43977
--tserver_master_addrs=127.29.194.62:46167
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19377
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.194.1
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115480851507-30472-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:14:11.870887 19377 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:14:11.871129 19377 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:14:11.877501 19410 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:14:11.877645 19377 server_base.cc:1034] running on GCE node
W20260807 15:14:11.877502 19415 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:14:11.877838 19411 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:14:11.878274 19377 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:14:11.878636 19377 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:14:11.879846 19377 hybrid_clock.cc:648] HybridClock initialized: now 1786115651879823 us; error 45 us; skew 500 ppm
Aug 07 15:14:11 dist-test-slave-42z9 krb5kdc[18425](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115651, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.194.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:14:11.883350 19377 init.cc:377] Logged in from keytab as kudu/127.29.194.1@KRBTEST.COM (short username kudu)
I20260807 15:14:11.884001 19377 webserver.cc:460] Webserver started at http://127.29.194.1:43977/ using document root <none> and password file <none>
I20260807 15:14:11.884276 19377 fs_manager.cc:359] Metadata directory not provided
I20260807 15:14:11.884362 19377 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:14:11.886425 19377 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260807 15:14:11.887490 19426 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:14:11.887717 19377 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 15:14:11.887817 19377 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115480851507-30472-0/minicluster-data/ts-0/data,/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115480851507-30472-0/minicluster-data/ts-0/wal
uuid: "8dc40eed26fb4a0982ceb06e101d137e"
format_stamp: "Formatted at 2026-08-07 15:14:10 on dist-test-slave-42z9"
server_key: "1df099ade868588b8d17cd7196feb163"
server_key_iv: "04d1e05f9c728096baca1337f00dfa74"
server_key_version: "encryptionkey@0"
I20260807 15:14:11.888222 19377 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115480851507-30472-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115480851507-30472-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115480851507-30472-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:14:11.900753 19377 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:14:11.903019 19377 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:14:11.903169 19377 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:14:11.903427 19377 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:14:11.903921 19439 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260807 15:14:11.905956 19377 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260807 15:14:11.906024 19377 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260807 15:14:11.906062 19377 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260807 15:14:11.906737 19377 ts_tablet_manager.cc:610] Registered 1 tablets
I20260807 15:14:11.906773 19377 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260807 15:14:11.906893 19439 tablet_bootstrap.cc:492] T 44ab56248a404a3a9412aa8f1d056e78 P 8dc40eed26fb4a0982ceb06e101d137e: Bootstrap starting.
I20260807 15:14:11.913691 19377 rpc_server.cc:307] RPC server started. Bound to: 127.29.194.1:43899
I20260807 15:14:11.913749 19593 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.194.1:43899 every 8 connection(s)
I20260807 15:14:11.914168 19377 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115480851507-30472-0/minicluster-data/ts-0/data/info.pb
I20260807 15:14:11.918465 19439 log.cc:824] T 44ab56248a404a3a9412aa8f1d056e78 P 8dc40eed26fb4a0982ceb06e101d137e: Log is configured to *not* fsync() on all Append() calls
I20260807 15:14:11.921486 19439 tablet_bootstrap.cc:492] T 44ab56248a404a3a9412aa8f1d056e78 P 8dc40eed26fb4a0982ceb06e101d137e: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 15:14:11.921813 19439 tablet_bootstrap.cc:492] T 44ab56248a404a3a9412aa8f1d056e78 P 8dc40eed26fb4a0982ceb06e101d137e: Bootstrap complete.
I20260807 15:14:11.923022 19439 ts_tablet_manager.cc:1397] T 44ab56248a404a3a9412aa8f1d056e78 P 8dc40eed26fb4a0982ceb06e101d137e: Time spent bootstrapping tablet: real 0.016s user 0.009s sys 0.005s
I20260807 15:14:11.923799 30472 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7fZSIL/build/release/bin/kudu as pid 19377
I20260807 15:14:11.924086 30472 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7fZSIL/build/release/bin/kudu
/tmp/dist-test-task7fZSIL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115480851507-30472-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115480851507-30472-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115480851507-30472-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115480851507-30472-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.194.2:33663
--local_ip_for_outbound_sockets=127.29.194.2
--tserver_master_addrs=127.29.194.62:46167
--webserver_port=36331
--webserver_interface=127.29.194.2
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.2.keytab
--principal=kudu/127.29.194.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.194.20:43617
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115480851507-30472-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115480851507-30472-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115480851507-30472-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 15:14:11.924811 19439 raft_consensus.cc:348] T 44ab56248a404a3a9412aa8f1d056e78 P 8dc40eed26fb4a0982ceb06e101d137e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff3aa95770c3407f9a7ec35fe5b235c2" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 33663 } } peers { permanent_uuid: "8dc40eed26fb4a0982ceb06e101d137e" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 43899 } } peers { permanent_uuid: "887081dc41e341b7911a7061f5f6d3a1" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 34127 } }
I20260807 15:14:11.925180 19439 raft_consensus.cc:729] T 44ab56248a404a3a9412aa8f1d056e78 P 8dc40eed26fb4a0982ceb06e101d137e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8dc40eed26fb4a0982ceb06e101d137e, State: Initialized, Role: FOLLOWER
I20260807 15:14:11.925321 19439 consensus_queue.cc:260] T 44ab56248a404a3a9412aa8f1d056e78 P 8dc40eed26fb4a0982ceb06e101d137e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff3aa95770c3407f9a7ec35fe5b235c2" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 33663 } } peers { permanent_uuid: "8dc40eed26fb4a0982ceb06e101d137e" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 43899 } } peers { permanent_uuid: "887081dc41e341b7911a7061f5f6d3a1" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 34127 } }
I20260807 15:14:11.925645 19439 ts_tablet_manager.cc:1428] T 44ab56248a404a3a9412aa8f1d056e78 P 8dc40eed26fb4a0982ceb06e101d137e: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
Aug 07 15:14:11 dist-test-slave-42z9 krb5kdc[18425](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115651, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.194.1@KRBTEST.COM for kudu/127.29.194.62@KRBTEST.COM
I20260807 15:14:11.930932 19596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:11.914912 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:46167 (local address 127.29.194.1:49179)
0807 15:14:11.919126 (+ 4214us) negotiation.cc:107] Waiting for socket to connect
0807 15:14:11.919166 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0807 15:14:11.919721 (+ 555us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:14:11.920201 (+ 480us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:14:11.920205 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:14:11.920485 (+ 280us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:14:11.920675 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:14:11.920680 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:14:11.922034 (+ 1354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:14:11.922036 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:14:11.922878 (+ 842us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:14:11.922881 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:14:11.923527 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:14:11.923529 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:14:11.923706 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:11.924440 (+ 734us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:14:11.924451 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:14:11.926758 (+ 2307us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:14:11.929019 (+ 2261us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:14:11.929027 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:14:11.929029 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:14:11.929332 (+ 303us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:14:11.929667 (+ 335us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:14:11.929668 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:14:11.929668 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:14:11.929800 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:14:11.930272 (+ 472us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:14:11.930277 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:14:11.930431 (+ 154us) client_negotiation.cc:769] Sending connection context
0807 15:14:11.930534 (+ 103us) client_negotiation.cc:240] Negotiation successful
0807 15:14:11.930704 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4171,"thread_start_us":110,"threads_started":1}
I20260807 15:14:11.930970 19597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:11.915025 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.194.1:49179 (local address 127.29.194.62:46167)
0807 15:14:11.919091 (+ 4066us) server_negotiation.cc:206] Beginning negotiation
0807 15:14:11.919104 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 15:14:11.919310 (+ 206us) server_negotiation.cc:407] Connection header received
0807 15:14:11.919946 (+ 636us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:14:11.919953 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:14:11.920009 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:14:11.920130 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:14:11.920799 (+ 669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:11.921889 (+ 1090us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:11.922971 (+ 1082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:11.923408 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:11.923510 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:11.926888 (+ 3378us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:14:11.926893 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:14:11.926894 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:14:11.926907 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:14:11.928906 (+ 1999us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:14:11.929430 (+ 524us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:14:11.929434 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:14:11.929434 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:14:11.929595 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:14:11.929872 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:14:11.929872 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:14:11.929873 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:14:11.930201 (+ 328us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:14:11.930292 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0807 15:14:11.930617 (+ 325us) server_negotiation.cc:299] Negotiation successful
0807 15:14:11.930769 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3990,"thread_start_us":76,"threads_started":1}
I20260807 15:14:11.932327 19594 heartbeater.cc:347] Connected to a master server at 127.29.194.62:46167
I20260807 15:14:11.932436 19594 heartbeater.cc:464] Registering TS with master...
I20260807 15:14:11.932626 19594 heartbeater.cc:511] Master 127.29.194.62:46167 requested a full tablet report, sending...
I20260807 15:14:11.933513 19320 ts_manager.cc:194] Registered new tserver with Master: 8dc40eed26fb4a0982ceb06e101d137e (127.29.194.1:43899)
I20260807 15:14:11.934824 19320 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.194.1@KRBTEST.COM'} at 127.29.194.1:49179
I20260807 15:14:11.935848 19594 heartbeater.cc:503] Master 127.29.194.62:46167 was elected leader, sending a full tablet report...
Aug 07 15:14:11 dist-test-slave-42z9 krb5kdc[18425](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115651, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.194.62@KRBTEST.COM for kudu/127.29.194.1@KRBTEST.COM
I20260807 15:14:11.944888 19602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:11.934213 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.1:43899 (local address 127.0.0.1:50472)
0807 15:14:11.934646 (+ 433us) negotiation.cc:107] Waiting for socket to connect
0807 15:14:11.934682 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0807 15:14:11.934746 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:14:11.935058 (+ 312us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:14:11.935062 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:14:11.935636 (+ 574us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:14:11.935737 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:14:11.935740 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:14:11.937263 (+ 1523us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:14:11.937265 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:14:11.938234 (+ 969us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:14:11.938237 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:14:11.938920 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:14:11.938920 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:14:11.939019 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:11.939625 (+ 606us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:14:11.939631 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:14:11.941518 (+ 1887us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:14:11.943586 (+ 2068us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:14:11.943594 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:14:11.943596 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:14:11.943869 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:14:11.944094 (+ 225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:14:11.944095 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:14:11.944095 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:14:11.944162 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:14:11.944490 (+ 328us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:14:11.944496 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:14:11.944555 (+ 59us) client_negotiation.cc:769] Sending connection context
0807 15:14:11.944638 (+ 83us) client_negotiation.cc:240] Negotiation successful
0807 15:14:11.944769 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":402,"thread_start_us":114,"threads_started":1}
I20260807 15:14:11.944918 19604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:11.934490 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50472 (local address 127.29.194.1:43899)
0807 15:14:11.934876 (+ 386us) server_negotiation.cc:206] Beginning negotiation
0807 15:14:11.934881 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:14:11.934890 (+ 9us) server_negotiation.cc:407] Connection header received
0807 15:14:11.934923 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:14:11.934926 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:14:11.934939 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:14:11.934993 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:14:11.935832 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:11.937160 (+ 1328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:11.938334 (+ 1174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:11.938822 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:11.938910 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:11.941632 (+ 2722us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:14:11.941645 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:14:11.941645 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:14:11.941654 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:14:11.943490 (+ 1836us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:14:11.943951 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:14:11.943955 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:14:11.943956 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:14:11.944029 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:14:11.944218 (+ 189us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:14:11.944219 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:14:11.944219 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:14:11.944416 (+ 197us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:14:11.944520 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0807 15:14:11.944696 (+ 176us) server_negotiation.cc:299] Negotiation successful
0807 15:14:11.944805 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":343,"thread_start_us":69,"threads_started":1}
I20260807 15:14:11.945911 19488 tablet_service.cc:1514] Processing DeleteTablet for tablet 44ab56248a404a3a9412aa8f1d056e78 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:14:11 UTC) from {username='kudu', principal='kudu/127.29.194.62@KRBTEST.COM'} at 127.0.0.1:50472
I20260807 15:14:11.946657 19605 tablet_replica.cc:330] stopping tablet replica
I20260807 15:14:11.946774 19605 raft_consensus.cc:2229] T 44ab56248a404a3a9412aa8f1d056e78 P 8dc40eed26fb4a0982ceb06e101d137e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 15:14:11.946842 19605 raft_consensus.cc:2258] T 44ab56248a404a3a9412aa8f1d056e78 P 8dc40eed26fb4a0982ceb06e101d137e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:14:11.947191 19605 ts_tablet_manager.cc:1905] T 44ab56248a404a3a9412aa8f1d056e78 P 8dc40eed26fb4a0982ceb06e101d137e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:14:11.949564 19605 ts_tablet_manager.cc:1918] T 44ab56248a404a3a9412aa8f1d056e78 P 8dc40eed26fb4a0982ceb06e101d137e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 15:14:11.949640 19605 log.cc:1196] T 44ab56248a404a3a9412aa8f1d056e78 P 8dc40eed26fb4a0982ceb06e101d137e: Deleting WAL directory at /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115480851507-30472-0/minicluster-data/ts-0/wal/wals/44ab56248a404a3a9412aa8f1d056e78
I20260807 15:14:11.949851 19605 ts_tablet_manager.cc:1939] T 44ab56248a404a3a9412aa8f1d056e78 P 8dc40eed26fb4a0982ceb06e101d137e: Deleting consensus metadata
I20260807 15:14:11.950371 19295 catalog_manager.cc:4841] TS 8dc40eed26fb4a0982ceb06e101d137e (127.29.194.1:43899): tablet 44ab56248a404a3a9412aa8f1d056e78 (table test-table [id=cab4e6f902554161ac8dd5d1920b80b6]) successfully deleted
W20260807 15:14:12.014554 19601 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:14:12.014791 19601 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:14:12.014825 19601 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:14:12.016080 19601 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:14:12.016134 19601 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:14:12.016171 19601 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.194.2
I20260807 15:14:12.018963 19601 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.194.20:43617
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115480851507-30472-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115480851507-30472-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.2.keytab
--principal=kudu/127.29.194.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.194.2:33663
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115480851507-30472-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.194.2
--webserver_port=36331
--tserver_master_addrs=127.29.194.62:46167
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19601
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.194.2
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115480851507-30472-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:14:12.019454 19601 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:14:12.019974 19601 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:14:12.025591 19613 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:14:12.025523 19617 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:14:12.025553 19614 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:14:12.025709 19601 server_base.cc:1034] running on GCE node
I20260807 15:14:12.026047 19601 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:14:12.026296 19601 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:14:12.027478 19601 hybrid_clock.cc:648] HybridClock initialized: now 1786115652027460 us; error 30 us; skew 500 ppm
Aug 07 15:14:12 dist-test-slave-42z9 krb5kdc[18425](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115652, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.194.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:14:12.030236 19601 init.cc:377] Logged in from keytab as kudu/127.29.194.2@KRBTEST.COM (short username kudu)
I20260807 15:14:12.030790 19601 webserver.cc:460] Webserver started at http://127.29.194.2:36331/ using document root <none> and password file <none>
I20260807 15:14:12.031076 19601 fs_manager.cc:359] Metadata directory not provided
I20260807 15:14:12.031148 19601 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:14:12.032575 19601 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260807 15:14:12.033500 19625 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:14:12.033740 19601 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 15:14:12.033842 19601 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115480851507-30472-0/minicluster-data/ts-1/data,/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115480851507-30472-0/minicluster-data/ts-1/wal
uuid: "ff3aa95770c3407f9a7ec35fe5b235c2"
format_stamp: "Formatted at 2026-08-07 15:14:11 on dist-test-slave-42z9"
server_key: "2138424ff93118e26d34ccb996845cb7"
server_key_iv: "7ea05b46921a36dc1c9400216469249f"
server_key_version: "encryptionkey@0"
I20260807 15:14:12.034215 19601 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115480851507-30472-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115480851507-30472-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115480851507-30472-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:14:12.058873 19601 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:14:12.061120 19601 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:14:12.061269 19601 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:14:12.061568 19601 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:14:12.061913 19601 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:14:12.061949 19601 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:14:12.061978 19601 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:14:12.061995 19601 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:14:12.068846 19601 rpc_server.cc:307] RPC server started. Bound to: 127.29.194.2:33663
I20260807 15:14:12.068949 19797 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.194.2:33663 every 8 connection(s)
I20260807 15:14:12.069296 19601 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115480851507-30472-0/minicluster-data/ts-1/data/info.pb
I20260807 15:14:12.071710 30472 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7fZSIL/build/release/bin/kudu as pid 19601
I20260807 15:14:12.071990 30472 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7fZSIL/build/release/bin/kudu
/tmp/dist-test-task7fZSIL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115480851507-30472-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115480851507-30472-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115480851507-30472-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115480851507-30472-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.194.3:34127
--local_ip_for_outbound_sockets=127.29.194.3
--tserver_master_addrs=127.29.194.62:46167
--webserver_port=32813
--webserver_interface=127.29.194.3
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.3.keytab
--principal=kudu/127.29.194.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.194.20:43617
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115480851507-30472-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115480851507-30472-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115480851507-30472-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 15:14:12 dist-test-slave-42z9 krb5kdc[18425](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115652, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.194.2@KRBTEST.COM for kudu/127.29.194.62@KRBTEST.COM
I20260807 15:14:12.080225 19597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:12.070118 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.194.2:53825 (local address 127.29.194.62:46167)
0807 15:14:12.070200 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0807 15:14:12.070202 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:14:12.070398 (+ 196us) server_negotiation.cc:407] Connection header received
0807 15:14:12.070933 (+ 535us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:14:12.070934 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:14:12.070947 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:14:12.071011 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:14:12.072180 (+ 1169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:12.072854 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:12.074191 (+ 1337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:12.074543 (+ 352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:12.074628 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:12.077188 (+ 2560us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:14:12.077190 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:14:12.077190 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:14:12.077197 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:14:12.078658 (+ 1461us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:14:12.079181 (+ 523us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:14:12.079182 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:14:12.079183 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:14:12.079240 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:14:12.079563 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:14:12.079564 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:14:12.079564 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:14:12.079678 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:14:12.079724 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0807 15:14:12.080074 (+ 350us) server_negotiation.cc:299] Negotiation successful
0807 15:14:12.080156 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260807 15:14:12.080497 19800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:12.070038 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:46167 (local address 127.29.194.2:53825)
0807 15:14:12.070270 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0807 15:14:12.070314 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0807 15:14:12.070803 (+ 489us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:14:12.071080 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:14:12.071083 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:14:12.071764 (+ 681us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:14:12.072057 (+ 293us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:14:12.072065 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:14:12.072974 (+ 909us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:14:12.072975 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:14:12.074101 (+ 1126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:14:12.074104 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:14:12.074648 (+ 544us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:14:12.074649 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:14:12.074763 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:12.075325 (+ 562us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:14:12.075334 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:14:12.077042 (+ 1708us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:14:12.078751 (+ 1709us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:14:12.078760 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:14:12.078762 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:14:12.079093 (+ 331us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:14:12.079323 (+ 230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:14:12.079324 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:14:12.079325 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:14:12.079492 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:14:12.079739 (+ 247us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:14:12.079746 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:14:12.079927 (+ 181us) client_negotiation.cc:769] Sending connection context
0807 15:14:12.080061 (+ 134us) client_negotiation.cc:240] Negotiation successful
0807 15:14:12.080252 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":76,"threads_started":1}
I20260807 15:14:12.081048 19798 heartbeater.cc:347] Connected to a master server at 127.29.194.62:46167
I20260807 15:14:12.081151 19798 heartbeater.cc:464] Registering TS with master...
I20260807 15:14:12.081337 19798 heartbeater.cc:511] Master 127.29.194.62:46167 requested a full tablet report, sending...
I20260807 15:14:12.081847 19320 ts_manager.cc:194] Registered new tserver with Master: ff3aa95770c3407f9a7ec35fe5b235c2 (127.29.194.2:33663)
I20260807 15:14:12.082232 19320 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.194.2@KRBTEST.COM'} at 127.29.194.2:53825
W20260807 15:14:12.159176 19802 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:14:12.159438 19802 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:14:12.159466 19802 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:14:12.160844 19802 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:14:12.160908 19802 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:14:12.160938 19802 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.194.3
I20260807 15:14:12.162575 19802 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.194.20:43617
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115480851507-30472-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115480851507-30472-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.3.keytab
--principal=kudu/127.29.194.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.194.3:34127
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115480851507-30472-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.194.3
--webserver_port=32813
--tserver_master_addrs=127.29.194.62:46167
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19802
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.194.3
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115480851507-30472-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:14:12.162810 19802 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:14:12.163020 19802 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:14:12.168424 19817 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:14:12.168497 19802 server_base.cc:1034] running on GCE node
W20260807 15:14:12.168351 19820 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:14:12.168355 19816 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:14:12.168809 19802 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:14:12.169071 19802 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:14:12.170287 19802 hybrid_clock.cc:648] HybridClock initialized: now 1786115652170264 us; error 35 us; skew 500 ppm
Aug 07 15:14:12 dist-test-slave-42z9 krb5kdc[18425](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115652, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.194.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:14:12.172971 19802 init.cc:377] Logged in from keytab as kudu/127.29.194.3@KRBTEST.COM (short username kudu)
I20260807 15:14:12.173547 19802 webserver.cc:460] Webserver started at http://127.29.194.3:32813/ using document root <none> and password file <none>
I20260807 15:14:12.173780 19802 fs_manager.cc:359] Metadata directory not provided
I20260807 15:14:12.173846 19802 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:14:12.175585 19802 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:14:12.176554 19829 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:14:12.176903 19802 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:14:12.177045 19802 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115480851507-30472-0/minicluster-data/ts-2/data,/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115480851507-30472-0/minicluster-data/ts-2/wal
uuid: "887081dc41e341b7911a7061f5f6d3a1"
format_stamp: "Formatted at 2026-08-07 15:14:11 on dist-test-slave-42z9"
server_key: "2447b01858ab937b1461a913272b57a5"
server_key_iv: "b75c3f8a4ad4573456a01f06f0fa49a4"
server_key_version: "encryptionkey@0"
I20260807 15:14:12.177616 19802 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115480851507-30472-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115480851507-30472-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115480851507-30472-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:14:12.188531 19802 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:14:12.191053 19802 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:14:12.191205 19802 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:14:12.191445 19802 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:14:12.191785 19802 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:14:12.191829 19802 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:14:12.191859 19802 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:14:12.191886 19802 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:14:12.198082 19802 rpc_server.cc:307] RPC server started. Bound to: 127.29.194.3:34127
I20260807 15:14:12.198179 19996 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.194.3:34127 every 8 connection(s)
I20260807 15:14:12.198500 19802 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115480851507-30472-0/minicluster-data/ts-2/data/info.pb
I20260807 15:14:12.198822 30472 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7fZSIL/build/release/bin/kudu as pid 19802
Aug 07 15:14:12 dist-test-slave-42z9 krb5kdc[18425](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115652, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.194.3@KRBTEST.COM for kudu/127.29.194.62@KRBTEST.COM
I20260807 15:14:12.208174 20003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:12.199320 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37522 (local address 127.29.194.62:46167)
0807 15:14:12.199485 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0807 15:14:12.199486 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 15:14:12.200357 (+ 871us) server_negotiation.cc:407] Connection header received
0807 15:14:12.200382 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:14:12.200382 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:14:12.200395 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:14:12.200447 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:14:12.201266 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:12.202031 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:12.203190 (+ 1159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:12.203406 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:12.203465 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:12.204808 (+ 1343us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:14:12.204810 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:14:12.204810 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:14:12.204818 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:14:12.206849 (+ 2031us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:14:12.207306 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:14:12.207308 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:14:12.207308 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:14:12.207370 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:14:12.207634 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:14:12.207635 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:14:12.207636 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:14:12.207780 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:14:12.207931 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0807 15:14:12.208032 (+ 101us) server_negotiation.cc:299] Negotiation successful
0807 15:14:12.208100 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":67,"server-negotiator.queue_time_us":152,"thread_start_us":54,"threads_started":1}
I20260807 15:14:12.209383 19597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:12.199311 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.194.3:55381 (local address 127.29.194.62:46167)
0807 15:14:12.199373 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0807 15:14:12.199376 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:14:12.199609 (+ 233us) server_negotiation.cc:407] Connection header received
0807 15:14:12.200155 (+ 546us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:14:12.200156 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:14:12.200167 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:14:12.200228 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:14:12.201502 (+ 1274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:12.202172 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:12.203533 (+ 1361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:12.203713 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:12.203769 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:12.206168 (+ 2399us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:14:12.206170 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:14:12.206170 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:14:12.206178 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:14:12.207737 (+ 1559us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:14:12.208296 (+ 559us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:14:12.208297 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:14:12.208297 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:14:12.208350 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:14:12.208631 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:14:12.208632 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:14:12.208632 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:14:12.208762 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:14:12.208829 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0807 15:14:12.209243 (+ 414us) server_negotiation.cc:299] Negotiation successful
0807 15:14:12.209318 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260807 15:14:12.209506 20002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:12.199269 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:46167 (local address 127.29.194.3:55381)
0807 15:14:12.199471 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0807 15:14:12.199516 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0807 15:14:12.200029 (+ 513us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:14:12.200384 (+ 355us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:14:12.200388 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:14:12.201142 (+ 754us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:14:12.201397 (+ 255us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:14:12.201403 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:14:12.202286 (+ 883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:14:12.202287 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:14:12.203430 (+ 1143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:14:12.203434 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:14:12.203804 (+ 370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:14:12.203805 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:14:12.203942 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:12.204421 (+ 479us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:14:12.204431 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:14:12.206065 (+ 1634us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:14:12.207834 (+ 1769us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:14:12.207845 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:14:12.207847 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:14:12.208177 (+ 330us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:14:12.208423 (+ 246us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:14:12.208424 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:14:12.208424 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:14:12.208565 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:14:12.208834 (+ 269us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:14:12.208842 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:14:12.209037 (+ 195us) client_negotiation.cc:769] Sending connection context
0807 15:14:12.209151 (+ 114us) client_negotiation.cc:240] Negotiation successful
0807 15:14:12.209292 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":129,"threads_started":1}
I20260807 15:14:12.209960 19998 heartbeater.cc:347] Connected to a master server at 127.29.194.62:46167
I20260807 15:14:12.210057 19998 heartbeater.cc:464] Registering TS with master...
I20260807 15:14:12.210215 19998 heartbeater.cc:511] Master 127.29.194.62:46167 requested a full tablet report, sending...
I20260807 15:14:12.210561 19320 ts_manager.cc:194] Registered new tserver with Master: 887081dc41e341b7911a7061f5f6d3a1 (127.29.194.3:34127)
I20260807 15:14:12.210924 19320 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.194.3@KRBTEST.COM'} at 127.29.194.3:55381
I20260807 15:14:12.221164 30472 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 15:14:12.224196 19597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:12.221637 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37532 (local address 127.29.194.62:46167)
0807 15:14:12.221699 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0807 15:14:12.221701 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:14:12.221833 (+ 132us) server_negotiation.cc:407] Connection header received
0807 15:14:12.221923 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:14:12.221924 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:14:12.221971 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:14:12.222034 (+ 63us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 15:14:12.222164 (+ 130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:12.222668 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:12.223458 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:12.223658 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:12.223713 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:12.223868 (+ 155us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 15:14:12.223977 (+ 109us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 15:14:12.224055 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0807 15:14:12.224093 (+ 38us) server_negotiation.cc:299] Negotiation successful
0807 15:14:12.224116 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260807 15:14:12.224540 19320 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:37532:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 15:14:12.226325 19320 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 15:14:12.228688 19488 tablet_service.cc:1467] Processing CreateTablet for tablet 492b00874a3a4c3e8be6e36e4789503c (DEFAULT_TABLE table=test-table [id=156182587b8e412285d0e618db1da7af]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:14:12.228921 19488 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 492b00874a3a4c3e8be6e36e4789503c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:14:12.230794 19439 tablet_bootstrap.cc:492] T 492b00874a3a4c3e8be6e36e4789503c P 8dc40eed26fb4a0982ceb06e101d137e: Bootstrap starting.
I20260807 15:14:12.231235 19439 tablet_bootstrap.cc:654] T 492b00874a3a4c3e8be6e36e4789503c P 8dc40eed26fb4a0982ceb06e101d137e: Neither blocks nor log segments found. Creating new log.
I20260807 15:14:12.232715 20005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:12.228417 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.3:34127 (local address 127.0.0.1:40370)
0807 15:14:12.228583 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0807 15:14:12.228592 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0807 15:14:12.228635 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:14:12.229224 (+ 589us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:14:12.229225 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:14:12.229227 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:14:12.229279 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:14:12.229280 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:14:12.230299 (+ 1019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:14:12.230300 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:14:12.231582 (+ 1282us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:14:12.231586 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:14:12.232411 (+ 825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:14:12.232412 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:14:12.232489 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:12.232516 (+ 27us) client_negotiation.cc:769] Sending connection context
0807 15:14:12.232579 (+ 63us) client_negotiation.cc:240] Negotiation successful
0807 15:14:12.232632 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"mutex_wait_us":34,"thread_start_us":103,"threads_started":1}
I20260807 15:14:12.232837 19439 tablet_bootstrap.cc:492] T 492b00874a3a4c3e8be6e36e4789503c P 8dc40eed26fb4a0982ceb06e101d137e: No bootstrap required, opened a new log
I20260807 15:14:12.232887 19439 ts_tablet_manager.cc:1397] T 492b00874a3a4c3e8be6e36e4789503c P 8dc40eed26fb4a0982ceb06e101d137e: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260807 15:14:12.232913 20008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:12.228693 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40370 (local address 127.29.194.3:34127)
0807 15:14:12.229069 (+ 376us) server_negotiation.cc:206] Beginning negotiation
0807 15:14:12.229073 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:14:12.229083 (+ 10us) server_negotiation.cc:407] Connection header received
0807 15:14:12.229118 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:14:12.229120 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:14:12.229132 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:14:12.229199 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:14:12.229340 (+ 141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:12.230197 (+ 857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:12.231677 (+ 1480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:12.232314 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:12.232380 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:12.232446 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0807 15:14:12.232698 (+ 252us) server_negotiation.cc:299] Negotiation successful
0807 15:14:12.232795 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":340,"thread_start_us":88,"threads_started":1}
I20260807 15:14:12.233040 19439 raft_consensus.cc:348] T 492b00874a3a4c3e8be6e36e4789503c P 8dc40eed26fb4a0982ceb06e101d137e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff3aa95770c3407f9a7ec35fe5b235c2" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 33663 } } peers { permanent_uuid: "887081dc41e341b7911a7061f5f6d3a1" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 34127 } } peers { permanent_uuid: "8dc40eed26fb4a0982ceb06e101d137e" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 43899 } }
I20260807 15:14:12.233114 19602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:12.228388 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.2:33663 (local address 127.0.0.1:59580)
0807 15:14:12.228438 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0807 15:14:12.228448 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 15:14:12.228483 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:14:12.228948 (+ 465us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:14:12.228949 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:14:12.228952 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:14:12.229008 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:14:12.229010 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:14:12.230425 (+ 1415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:14:12.230426 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:14:12.231636 (+ 1210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:14:12.231641 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:14:12.232801 (+ 1160us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:14:12.232803 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:14:12.232926 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:12.232936 (+ 10us) client_negotiation.cc:769] Sending connection context
0807 15:14:12.232969 (+ 33us) client_negotiation.cc:240] Negotiation successful
0807 15:14:12.233039 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32,"mutex_wait_us":60}
I20260807 15:14:12.233121 19439 raft_consensus.cc:374] T 492b00874a3a4c3e8be6e36e4789503c P 8dc40eed26fb4a0982ceb06e101d137e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:14:12.233183 19439 raft_consensus.cc:729] T 492b00874a3a4c3e8be6e36e4789503c P 8dc40eed26fb4a0982ceb06e101d137e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8dc40eed26fb4a0982ceb06e101d137e, State: Initialized, Role: FOLLOWER
I20260807 15:14:12.233244 19439 consensus_queue.cc:260] T 492b00874a3a4c3e8be6e36e4789503c P 8dc40eed26fb4a0982ceb06e101d137e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff3aa95770c3407f9a7ec35fe5b235c2" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 33663 } } peers { permanent_uuid: "887081dc41e341b7911a7061f5f6d3a1" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 34127 } } peers { permanent_uuid: "8dc40eed26fb4a0982ceb06e101d137e" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 43899 } }
I20260807 15:14:12.233336 19439 ts_tablet_manager.cc:1428] T 492b00874a3a4c3e8be6e36e4789503c P 8dc40eed26fb4a0982ceb06e101d137e: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260807 15:14:12.233609 20007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:12.228600 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59580 (local address 127.29.194.2:33663)
0807 15:14:12.228762 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0807 15:14:12.228767 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:14:12.228776 (+ 9us) server_negotiation.cc:407] Connection header received
0807 15:14:12.228817 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:14:12.228820 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:14:12.228835 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:14:12.228915 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:14:12.229105 (+ 190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:12.230273 (+ 1168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:12.231809 (+ 1536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:12.232529 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:12.233296 (+ 767us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:12.233389 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0807 15:14:12.233415 (+ 26us) server_negotiation.cc:299] Negotiation successful
0807 15:14:12.233518 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130,"thread_start_us":93,"threads_started":1}
I20260807 15:14:12.233814 19901 tablet_service.cc:1467] Processing CreateTablet for tablet 492b00874a3a4c3e8be6e36e4789503c (DEFAULT_TABLE table=test-table [id=156182587b8e412285d0e618db1da7af]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:14:12.234242 19901 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 492b00874a3a4c3e8be6e36e4789503c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:14:12.234414 19701 tablet_service.cc:1467] Processing CreateTablet for tablet 492b00874a3a4c3e8be6e36e4789503c (DEFAULT_TABLE table=test-table [id=156182587b8e412285d0e618db1da7af]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:14:12.234745 19701 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 492b00874a3a4c3e8be6e36e4789503c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:14:12.238777 20010 tablet_bootstrap.cc:492] T 492b00874a3a4c3e8be6e36e4789503c P ff3aa95770c3407f9a7ec35fe5b235c2: Bootstrap starting.
I20260807 15:14:12.239181 20012 tablet_bootstrap.cc:492] T 492b00874a3a4c3e8be6e36e4789503c P 887081dc41e341b7911a7061f5f6d3a1: Bootstrap starting.
I20260807 15:14:12.239598 20010 tablet_bootstrap.cc:654] T 492b00874a3a4c3e8be6e36e4789503c P ff3aa95770c3407f9a7ec35fe5b235c2: Neither blocks nor log segments found. Creating new log.
I20260807 15:14:12.239794 20012 tablet_bootstrap.cc:654] T 492b00874a3a4c3e8be6e36e4789503c P 887081dc41e341b7911a7061f5f6d3a1: Neither blocks nor log segments found. Creating new log.
I20260807 15:14:12.240145 20010 log.cc:824] T 492b00874a3a4c3e8be6e36e4789503c P ff3aa95770c3407f9a7ec35fe5b235c2: Log is configured to *not* fsync() on all Append() calls
I20260807 15:14:12.240260 20012 log.cc:824] T 492b00874a3a4c3e8be6e36e4789503c P 887081dc41e341b7911a7061f5f6d3a1: Log is configured to *not* fsync() on all Append() calls
I20260807 15:14:12.241048 20010 tablet_bootstrap.cc:492] T 492b00874a3a4c3e8be6e36e4789503c P ff3aa95770c3407f9a7ec35fe5b235c2: No bootstrap required, opened a new log
I20260807 15:14:12.241107 20010 ts_tablet_manager.cc:1397] T 492b00874a3a4c3e8be6e36e4789503c P ff3aa95770c3407f9a7ec35fe5b235c2: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260807 15:14:12.241272 20012 tablet_bootstrap.cc:492] T 492b00874a3a4c3e8be6e36e4789503c P 887081dc41e341b7911a7061f5f6d3a1: No bootstrap required, opened a new log
I20260807 15:14:12.241333 20012 ts_tablet_manager.cc:1397] T 492b00874a3a4c3e8be6e36e4789503c P 887081dc41e341b7911a7061f5f6d3a1: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 15:14:12.242587 20010 raft_consensus.cc:348] T 492b00874a3a4c3e8be6e36e4789503c P ff3aa95770c3407f9a7ec35fe5b235c2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff3aa95770c3407f9a7ec35fe5b235c2" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 33663 } } peers { permanent_uuid: "887081dc41e341b7911a7061f5f6d3a1" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 34127 } } peers { permanent_uuid: "8dc40eed26fb4a0982ceb06e101d137e" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 43899 } }
I20260807 15:14:12.242734 20010 raft_consensus.cc:374] T 492b00874a3a4c3e8be6e36e4789503c P ff3aa95770c3407f9a7ec35fe5b235c2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:14:12.242758 20010 raft_consensus.cc:729] T 492b00874a3a4c3e8be6e36e4789503c P ff3aa95770c3407f9a7ec35fe5b235c2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ff3aa95770c3407f9a7ec35fe5b235c2, State: Initialized, Role: FOLLOWER
I20260807 15:14:12.242727 20012 raft_consensus.cc:348] T 492b00874a3a4c3e8be6e36e4789503c P 887081dc41e341b7911a7061f5f6d3a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff3aa95770c3407f9a7ec35fe5b235c2" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 33663 } } peers { permanent_uuid: "887081dc41e341b7911a7061f5f6d3a1" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 34127 } } peers { permanent_uuid: "8dc40eed26fb4a0982ceb06e101d137e" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 43899 } }
I20260807 15:14:12.242890 20012 raft_consensus.cc:374] T 492b00874a3a4c3e8be6e36e4789503c P 887081dc41e341b7911a7061f5f6d3a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:14:12.242872 20010 consensus_queue.cc:260] T 492b00874a3a4c3e8be6e36e4789503c P ff3aa95770c3407f9a7ec35fe5b235c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff3aa95770c3407f9a7ec35fe5b235c2" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 33663 } } peers { permanent_uuid: "887081dc41e341b7911a7061f5f6d3a1" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 34127 } } peers { permanent_uuid: "8dc40eed26fb4a0982ceb06e101d137e" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 43899 } }
I20260807 15:14:12.242940 20012 raft_consensus.cc:729] T 492b00874a3a4c3e8be6e36e4789503c P 887081dc41e341b7911a7061f5f6d3a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 887081dc41e341b7911a7061f5f6d3a1, State: Initialized, Role: FOLLOWER
I20260807 15:14:12.243052 20012 consensus_queue.cc:260] T 492b00874a3a4c3e8be6e36e4789503c P 887081dc41e341b7911a7061f5f6d3a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff3aa95770c3407f9a7ec35fe5b235c2" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 33663 } } peers { permanent_uuid: "887081dc41e341b7911a7061f5f6d3a1" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 34127 } } peers { permanent_uuid: "8dc40eed26fb4a0982ceb06e101d137e" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 43899 } }
I20260807 15:14:12.243185 19798 heartbeater.cc:503] Master 127.29.194.62:46167 was elected leader, sending a full tablet report...
I20260807 15:14:12.243320 20012 ts_tablet_manager.cc:1428] T 492b00874a3a4c3e8be6e36e4789503c P 887081dc41e341b7911a7061f5f6d3a1: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260807 15:14:12.243396 19998 heartbeater.cc:503] Master 127.29.194.62:46167 was elected leader, sending a full tablet report...
I20260807 15:14:12.243176 20010 ts_tablet_manager.cc:1428] T 492b00874a3a4c3e8be6e36e4789503c P ff3aa95770c3407f9a7ec35fe5b235c2: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
W20260807 15:14:12.319978 19799 tablet.cc:2369] T 492b00874a3a4c3e8be6e36e4789503c P ff3aa95770c3407f9a7ec35fe5b235c2: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 15:14:12.420372 19595 tablet.cc:2369] T 492b00874a3a4c3e8be6e36e4789503c P 8dc40eed26fb4a0982ceb06e101d137e: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 15:14:12.449147 19999 tablet.cc:2369] T 492b00874a3a4c3e8be6e36e4789503c P 887081dc41e341b7911a7061f5f6d3a1: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 15:14:12.449882 20015 raft_consensus.cc:482] T 492b00874a3a4c3e8be6e36e4789503c P ff3aa95770c3407f9a7ec35fe5b235c2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 15:14:12.450029 20015 raft_consensus.cc:504] T 492b00874a3a4c3e8be6e36e4789503c P ff3aa95770c3407f9a7ec35fe5b235c2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff3aa95770c3407f9a7ec35fe5b235c2" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 33663 } } peers { permanent_uuid: "887081dc41e341b7911a7061f5f6d3a1" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 34127 } } peers { permanent_uuid: "8dc40eed26fb4a0982ceb06e101d137e" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 43899 } }
I20260807 15:14:12.450455 20015 leader_election.cc:288] T 492b00874a3a4c3e8be6e36e4789503c P ff3aa95770c3407f9a7ec35fe5b235c2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 887081dc41e341b7911a7061f5f6d3a1 (127.29.194.3:34127), 8dc40eed26fb4a0982ceb06e101d137e (127.29.194.1:43899)
I20260807 15:14:12.453368 20008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:12.450781 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.194.2:48719 (local address 127.29.194.3:34127)
0807 15:14:12.450824 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0807 15:14:12.450826 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:14:12.451004 (+ 178us) server_negotiation.cc:407] Connection header received
0807 15:14:12.451038 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:14:12.451039 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:14:12.451048 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:14:12.451097 (+ 49us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:14:12.451250 (+ 153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:12.451714 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:12.452648 (+ 934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:12.453038 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:12.453102 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:12.453112 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0807 15:14:12.453260 (+ 148us) server_negotiation.cc:299] Negotiation successful
0807 15:14:12.453299 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260807 15:14:12.453390 20018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:12.450687 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.3:34127 (local address 127.29.194.2:48719)
0807 15:14:12.450917 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0807 15:14:12.450926 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0807 15:14:12.450985 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:14:12.451114 (+ 129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:14:12.451115 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:14:12.451116 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:14:12.451184 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:14:12.451185 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:14:12.451798 (+ 613us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:14:12.451799 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:14:12.452549 (+ 750us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:14:12.452552 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:14:12.453124 (+ 572us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:14:12.453126 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:14:12.453189 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:12.453193 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 15:14:12.453239 (+ 46us) client_negotiation.cc:240] Negotiation successful
0807 15:14:12.453280 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":212,"mutex_wait_us":15,"thread_start_us":113,"threads_started":1}
I20260807 15:14:12.453420 20019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:12.450789 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.194.2:33641 (local address 127.29.194.1:43899)
0807 15:14:12.450903 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0807 15:14:12.450905 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:14:12.450913 (+ 8us) server_negotiation.cc:407] Connection header received
0807 15:14:12.450953 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:14:12.450954 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:14:12.450976 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:14:12.451042 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:14:12.451195 (+ 153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:12.451692 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:12.452640 (+ 948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:12.453074 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:12.453142 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:12.453215 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0807 15:14:12.453301 (+ 86us) server_negotiation.cc:299] Negotiation successful
0807 15:14:12.453351 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94,"thread_start_us":54,"threads_started":1}
I20260807 15:14:12.453639 19800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:12.450687 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.1:43899 (local address 127.29.194.2:33641)
0807 15:14:12.450760 (+ 73us) negotiation.cc:107] Waiting for socket to connect
0807 15:14:12.450775 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 15:14:12.450843 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:14:12.451059 (+ 216us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:14:12.451060 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:14:12.451062 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:14:12.451132 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:14:12.451134 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:14:12.451769 (+ 635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:14:12.451770 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:14:12.452548 (+ 778us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:14:12.452550 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:14:12.453152 (+ 602us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:14:12.453153 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:14:12.453235 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:12.453238 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 15:14:12.453288 (+ 50us) client_negotiation.cc:240] Negotiation successful
0807 15:14:12.453330 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":46,"mutex_wait_us":32}
I20260807 15:14:12.453797 19933 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "492b00874a3a4c3e8be6e36e4789503c" candidate_uuid: "ff3aa95770c3407f9a7ec35fe5b235c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "887081dc41e341b7911a7061f5f6d3a1" is_pre_election: true
I20260807 15:14:12.453974 19933 raft_consensus.cc:2454] T 492b00874a3a4c3e8be6e36e4789503c P 887081dc41e341b7911a7061f5f6d3a1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff3aa95770c3407f9a7ec35fe5b235c2 in term 0.
I20260807 15:14:12.453931 19530 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "492b00874a3a4c3e8be6e36e4789503c" candidate_uuid: "ff3aa95770c3407f9a7ec35fe5b235c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8dc40eed26fb4a0982ceb06e101d137e" is_pre_election: true
I20260807 15:14:12.454077 19530 raft_consensus.cc:2454] T 492b00874a3a4c3e8be6e36e4789503c P 8dc40eed26fb4a0982ceb06e101d137e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff3aa95770c3407f9a7ec35fe5b235c2 in term 0.
I20260807 15:14:12.454214 19635 leader_election.cc:302] T 492b00874a3a4c3e8be6e36e4789503c P ff3aa95770c3407f9a7ec35fe5b235c2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 887081dc41e341b7911a7061f5f6d3a1, ff3aa95770c3407f9a7ec35fe5b235c2; no voters:
I20260807 15:14:12.454360 20015 raft_consensus.cc:2781] T 492b00874a3a4c3e8be6e36e4789503c P ff3aa95770c3407f9a7ec35fe5b235c2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 15:14:12.454419 20015 raft_consensus.cc:482] T 492b00874a3a4c3e8be6e36e4789503c P ff3aa95770c3407f9a7ec35fe5b235c2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 15:14:12.454448 20015 raft_consensus.cc:3037] T 492b00874a3a4c3e8be6e36e4789503c P ff3aa95770c3407f9a7ec35fe5b235c2 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:14:12.455147 20015 raft_consensus.cc:504] T 492b00874a3a4c3e8be6e36e4789503c P ff3aa95770c3407f9a7ec35fe5b235c2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff3aa95770c3407f9a7ec35fe5b235c2" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 33663 } } peers { permanent_uuid: "887081dc41e341b7911a7061f5f6d3a1" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 34127 } } peers { permanent_uuid: "8dc40eed26fb4a0982ceb06e101d137e" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 43899 } }
I20260807 15:14:12.455283 20015 leader_election.cc:288] T 492b00874a3a4c3e8be6e36e4789503c P ff3aa95770c3407f9a7ec35fe5b235c2 [CANDIDATE]: Term 1 election: Requested vote from peers 887081dc41e341b7911a7061f5f6d3a1 (127.29.194.3:34127), 8dc40eed26fb4a0982ceb06e101d137e (127.29.194.1:43899)
I20260807 15:14:12.455557 19933 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "492b00874a3a4c3e8be6e36e4789503c" candidate_uuid: "ff3aa95770c3407f9a7ec35fe5b235c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "887081dc41e341b7911a7061f5f6d3a1"
I20260807 15:14:12.455583 19530 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "492b00874a3a4c3e8be6e36e4789503c" candidate_uuid: "ff3aa95770c3407f9a7ec35fe5b235c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8dc40eed26fb4a0982ceb06e101d137e"
I20260807 15:14:12.455755 19530 raft_consensus.cc:3037] T 492b00874a3a4c3e8be6e36e4789503c P 8dc40eed26fb4a0982ceb06e101d137e [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:14:12.455750 19933 raft_consensus.cc:3037] T 492b00874a3a4c3e8be6e36e4789503c P 887081dc41e341b7911a7061f5f6d3a1 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:14:12.456918 19933 raft_consensus.cc:2454] T 492b00874a3a4c3e8be6e36e4789503c P 887081dc41e341b7911a7061f5f6d3a1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff3aa95770c3407f9a7ec35fe5b235c2 in term 1.
I20260807 15:14:12.456918 19530 raft_consensus.cc:2454] T 492b00874a3a4c3e8be6e36e4789503c P 8dc40eed26fb4a0982ceb06e101d137e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff3aa95770c3407f9a7ec35fe5b235c2 in term 1.
I20260807 15:14:12.457166 19635 leader_election.cc:302] T 492b00874a3a4c3e8be6e36e4789503c P ff3aa95770c3407f9a7ec35fe5b235c2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 887081dc41e341b7911a7061f5f6d3a1, ff3aa95770c3407f9a7ec35fe5b235c2; no voters:
I20260807 15:14:12.457294 20015 raft_consensus.cc:2781] T 492b00874a3a4c3e8be6e36e4789503c P ff3aa95770c3407f9a7ec35fe5b235c2 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:14:12.457551 20015 raft_consensus.cc:686] T 492b00874a3a4c3e8be6e36e4789503c P ff3aa95770c3407f9a7ec35fe5b235c2 [term 1 LEADER]: Becoming Leader. State: Replica: ff3aa95770c3407f9a7ec35fe5b235c2, State: Running, Role: LEADER
I20260807 15:14:12.457664 20015 consensus_queue.cc:237] T 492b00874a3a4c3e8be6e36e4789503c P ff3aa95770c3407f9a7ec35fe5b235c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff3aa95770c3407f9a7ec35fe5b235c2" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 33663 } } peers { permanent_uuid: "887081dc41e341b7911a7061f5f6d3a1" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 34127 } } peers { permanent_uuid: "8dc40eed26fb4a0982ceb06e101d137e" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 43899 } }
I20260807 15:14:12.458567 19318 catalog_manager.cc:5495] T 492b00874a3a4c3e8be6e36e4789503c P ff3aa95770c3407f9a7ec35fe5b235c2 reported cstate change: term changed from 0 to 1, leader changed from <none> to ff3aa95770c3407f9a7ec35fe5b235c2 (127.29.194.2). New cstate: current_term: 1 leader_uuid: "ff3aa95770c3407f9a7ec35fe5b235c2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff3aa95770c3407f9a7ec35fe5b235c2" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 33663 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "887081dc41e341b7911a7061f5f6d3a1" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 34127 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8dc40eed26fb4a0982ceb06e101d137e" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 43899 } health_report { overall_health: UNKNOWN } } }
I20260807 15:14:12.466351 20007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:12.463970 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59592 (local address 127.29.194.2:33663)
0807 15:14:12.464028 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0807 15:14:12.464030 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:14:12.464039 (+ 9us) server_negotiation.cc:407] Connection header received
0807 15:14:12.464115 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:14:12.464116 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:14:12.464138 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:14:12.464204 (+ 66us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 15:14:12.464327 (+ 123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:12.464809 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:12.465539 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:12.465748 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:12.465803 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:12.465949 (+ 146us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 15:14:12.466056 (+ 107us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 15:14:12.466128 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0807 15:14:12.466166 (+ 38us) server_negotiation.cc:299] Negotiation successful
0807 15:14:12.466189 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260807 15:14:12.468233 19530 raft_consensus.cc:1261] T 492b00874a3a4c3e8be6e36e4789503c P 8dc40eed26fb4a0982ceb06e101d137e [term 1 FOLLOWER]: Refusing update from remote peer ff3aa95770c3407f9a7ec35fe5b235c2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:14:12.468593 19933 raft_consensus.cc:1261] T 492b00874a3a4c3e8be6e36e4789503c P 887081dc41e341b7911a7061f5f6d3a1 [term 1 FOLLOWER]: Refusing update from remote peer ff3aa95770c3407f9a7ec35fe5b235c2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:14:12.468658 20021 consensus_queue.cc:1035] T 492b00874a3a4c3e8be6e36e4789503c P ff3aa95770c3407f9a7ec35fe5b235c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8dc40eed26fb4a0982ceb06e101d137e" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 43899 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:14:12.468853 20021 consensus_queue.cc:1035] T 492b00874a3a4c3e8be6e36e4789503c P ff3aa95770c3407f9a7ec35fe5b235c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "887081dc41e341b7911a7061f5f6d3a1" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 34127 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:14:12.472429 20025 mvcc.cc:205] Tried to move back new op lower bound from 7315929712507080704 to 7315929712467095552. Current Snapshot: MvccSnapshot[applied={T|T < 7315929712507080704 or (T in {7315929712507080704})}]
I20260807 15:14:12.473124 19319 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:37532:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 15:14:12.473227 19319 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:37532:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 15:14:12.474062 19319 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 25474f26b3ab4143b43529dc11548fe2: Sending DeleteTablet for 3 replicas of tablet 492b00874a3a4c3e8be6e36e4789503c
I20260807 15:14:12.474347 19488 tablet_service.cc:1514] Processing DeleteTablet for tablet 492b00874a3a4c3e8be6e36e4789503c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:14:12 UTC) from {username='kudu', principal='kudu/127.29.194.62@KRBTEST.COM'} at 127.0.0.1:50472
I20260807 15:14:12.474422 19701 tablet_service.cc:1514] Processing DeleteTablet for tablet 492b00874a3a4c3e8be6e36e4789503c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:14:12 UTC) from {username='kudu', principal='kudu/127.29.194.62@KRBTEST.COM'} at 127.0.0.1:59580
I20260807 15:14:12.474493 19901 tablet_service.cc:1514] Processing DeleteTablet for tablet 492b00874a3a4c3e8be6e36e4789503c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:14:12 UTC) from {username='kudu', principal='kudu/127.29.194.62@KRBTEST.COM'} at 127.0.0.1:40370
I20260807 15:14:12.474658 20038 tablet_replica.cc:330] T 492b00874a3a4c3e8be6e36e4789503c P ff3aa95770c3407f9a7ec35fe5b235c2: stopping tablet replica
I20260807 15:14:12.474731 30472 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7fZSIL/build/release/bin/kudu with pid 19377
I20260807 15:14:12.474761 20038 raft_consensus.cc:2229] T 492b00874a3a4c3e8be6e36e4789503c P ff3aa95770c3407f9a7ec35fe5b235c2 [term 1 LEADER]: Raft consensus shutting down.
I20260807 15:14:12.474866 20038 raft_consensus.cc:2258] T 492b00874a3a4c3e8be6e36e4789503c P ff3aa95770c3407f9a7ec35fe5b235c2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:14:12.475005 20040 tablet_replica.cc:330] T 492b00874a3a4c3e8be6e36e4789503c P 887081dc41e341b7911a7061f5f6d3a1: stopping tablet replica
I20260807 15:14:12.475088 20040 raft_consensus.cc:2229] T 492b00874a3a4c3e8be6e36e4789503c P 887081dc41e341b7911a7061f5f6d3a1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 15:14:12.475180 20040 raft_consensus.cc:2258] T 492b00874a3a4c3e8be6e36e4789503c P 887081dc41e341b7911a7061f5f6d3a1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:14:12.490602 20038 ts_tablet_manager.cc:1905] T 492b00874a3a4c3e8be6e36e4789503c P ff3aa95770c3407f9a7ec35fe5b235c2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:14:12.492507 20038 ts_tablet_manager.cc:1918] T 492b00874a3a4c3e8be6e36e4789503c P ff3aa95770c3407f9a7ec35fe5b235c2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 15:14:12.492712 20038 log.cc:1196] T 492b00874a3a4c3e8be6e36e4789503c P ff3aa95770c3407f9a7ec35fe5b235c2: Deleting WAL directory at /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115480851507-30472-0/minicluster-data/ts-1/wal/wals/492b00874a3a4c3e8be6e36e4789503c
I20260807 15:14:12.493125 20038 ts_tablet_manager.cc:1939] T 492b00874a3a4c3e8be6e36e4789503c P ff3aa95770c3407f9a7ec35fe5b235c2: Deleting consensus metadata
I20260807 15:14:12.493786 19297 catalog_manager.cc:4841] TS ff3aa95770c3407f9a7ec35fe5b235c2 (127.29.194.2:33663): tablet 492b00874a3a4c3e8be6e36e4789503c (table test-table [id=156182587b8e412285d0e618db1da7af]) successfully deleted
I20260807 15:14:12.494566 20040 ts_tablet_manager.cc:1905] T 492b00874a3a4c3e8be6e36e4789503c P 887081dc41e341b7911a7061f5f6d3a1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:14:12.496330 20040 ts_tablet_manager.cc:1918] T 492b00874a3a4c3e8be6e36e4789503c P 887081dc41e341b7911a7061f5f6d3a1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 15:14:12.496490 20040 log.cc:1196] T 492b00874a3a4c3e8be6e36e4789503c P 887081dc41e341b7911a7061f5f6d3a1: Deleting WAL directory at /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115480851507-30472-0/minicluster-data/ts-2/wal/wals/492b00874a3a4c3e8be6e36e4789503c
I20260807 15:14:12.496963 20040 ts_tablet_manager.cc:1939] T 492b00874a3a4c3e8be6e36e4789503c P 887081dc41e341b7911a7061f5f6d3a1: Deleting consensus metadata
I20260807 15:14:12.499384 19296 catalog_manager.cc:4841] TS 887081dc41e341b7911a7061f5f6d3a1 (127.29.194.3:34127): tablet 492b00874a3a4c3e8be6e36e4789503c (table test-table [id=156182587b8e412285d0e618db1da7af]) successfully deleted
W20260807 15:14:12.504866 19295 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.29.194.1:43899 (error 108)
I20260807 15:14:12.505268 30472 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7fZSIL/build/release/bin/kudu with pid 19601
I20260807 15:14:12.505386 19602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:12.505258 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.1:43899 (local address 127.0.0.1:50482)
0807 15:14:12.505312 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0807 15:14:12.505349 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.194.1:43899: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260807 15:14:12.511798 19295 catalog_manager.cc:4584] TS 8dc40eed26fb4a0982ceb06e101d137e (127.29.194.1:43899): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 492b00874a3a4c3e8be6e36e4789503c: Network error: Client connection negotiation failed: client connection to 127.29.194.1:43899: connect: Connection refused (error 111)
I20260807 15:14:12.516696 30472 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7fZSIL/build/release/bin/kudu with pid 19802
I20260807 15:14:12.533526 30472 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7fZSIL/build/release/bin/kudu with pid 19274
I20260807 15:14:12.547472 30472 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7fZSIL/build/release/bin/kudu
/tmp/dist-test-task7fZSIL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115480851507-30472-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115480851507-30472-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115480851507-30472-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115480851507-30472-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.194.62:46167
--webserver_interface=127.29.194.62
--webserver_port=42443
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.62.keytab
--principal=kudu/127.29.194.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.194.20:43617
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.194.62:46167
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115480851507-30472-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115480851507-30472-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115480851507-30472-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:14:12.636066 20045 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:14:12.636281 20045 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:14:12.636303 20045 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:14:12.637461 20045 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:14:12.637501 20045 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:14:12.637514 20045 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:14:12.637526 20045 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:14:12.637535 20045 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:14:12.639017 20045 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.194.20:43617
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115480851507-30472-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115480851507-30472-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.194.62:46167
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.62.keytab
--principal=kudu/127.29.194.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.194.62:46167
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115480851507-30472-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.194.62
--webserver_port=42443
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20045
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115480851507-30472-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:14:12.639227 20045 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:14:12.639448 20045 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:14:12.644837 20052 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:14:12.644912 20051 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:14:12.644912 20054 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:14:12.645102 20045 server_base.cc:1034] running on GCE node
I20260807 15:14:12.645413 20045 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:14:12.645753 20045 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:14:12.646960 20045 hybrid_clock.cc:648] HybridClock initialized: now 1786115652646945 us; error 29 us; skew 500 ppm
Aug 07 15:14:12 dist-test-slave-42z9 krb5kdc[18425](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115652, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.194.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:14:12.649590 20045 init.cc:377] Logged in from keytab as kudu/127.29.194.62@KRBTEST.COM (short username kudu)
I20260807 15:14:12.650090 20045 webserver.cc:460] Webserver started at http://127.29.194.62:42443/ using document root <none> and password file <none>
I20260807 15:14:12.650329 20045 fs_manager.cc:359] Metadata directory not provided
I20260807 15:14:12.650381 20045 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:14:12.651790 20045 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:14:12.652534 20062 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:14:12.652761 20045 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:14:12.652976 20045 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115480851507-30472-0/minicluster-data/master-0/data,/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115480851507-30472-0/minicluster-data/master-0/wal
uuid: "25474f26b3ab4143b43529dc11548fe2"
format_stamp: "Formatted at 2026-08-07 15:14:10 on dist-test-slave-42z9"
server_key: "5499c9bcc5ff9f5b2103e1aa0425b153"
server_key_iv: "12cdb2959cfffadfd57e8f3f376805c6"
server_key_version: "encryptionkey@0"
I20260807 15:14:12.653378 20045 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115480851507-30472-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115480851507-30472-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115480851507-30472-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:14:12.674959 20045 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:14:12.678018 20045 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:14:12.678210 20045 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:14:12.682432 20045 rpc_server.cc:307] RPC server started. Bound to: 127.29.194.62:46167
I20260807 15:14:12.682466 20138 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.194.62:46167 every 8 connection(s)
I20260807 15:14:12.682858 20045 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115480851507-30472-0/minicluster-data/master-0/data/info.pb
I20260807 15:14:12.684037 30472 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7fZSIL/build/release/bin/kudu as pid 20045
I20260807 15:14:12.684353 30472 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7fZSIL/build/release/bin/kudu
/tmp/dist-test-task7fZSIL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115480851507-30472-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115480851507-30472-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115480851507-30472-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115480851507-30472-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.194.1:43899
--local_ip_for_outbound_sockets=127.29.194.1
--tserver_master_addrs=127.29.194.62:46167
--webserver_port=43977
--webserver_interface=127.29.194.1
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.1.keytab
--principal=kudu/127.29.194.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.194.20:43617
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115480851507-30472-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115480851507-30472-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115480851507-30472-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 15:14:12.684765 20139 sys_catalog.cc:263] Verifying existing consensus state
I20260807 15:14:12.685640 20139 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 25474f26b3ab4143b43529dc11548fe2: Bootstrap starting.
I20260807 15:14:12.694725 20139 log.cc:824] T 00000000000000000000000000000000 P 25474f26b3ab4143b43529dc11548fe2: Log is configured to *not* fsync() on all Append() calls
I20260807 15:14:12.697218 20139 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 25474f26b3ab4143b43529dc11548fe2: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 15:14:12.697609 20139 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 25474f26b3ab4143b43529dc11548fe2: Bootstrap complete.
I20260807 15:14:12.699924 20139 raft_consensus.cc:348] T 00000000000000000000000000000000 P 25474f26b3ab4143b43529dc11548fe2 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25474f26b3ab4143b43529dc11548fe2" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 46167 } }
I20260807 15:14:12.700163 20139 raft_consensus.cc:729] T 00000000000000000000000000000000 P 25474f26b3ab4143b43529dc11548fe2 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 25474f26b3ab4143b43529dc11548fe2, State: Initialized, Role: FOLLOWER
I20260807 15:14:12.700296 20139 consensus_queue.cc:260] T 00000000000000000000000000000000 P 25474f26b3ab4143b43529dc11548fe2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25474f26b3ab4143b43529dc11548fe2" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 46167 } }
I20260807 15:14:12.700407 20139 raft_consensus.cc:388] T 00000000000000000000000000000000 P 25474f26b3ab4143b43529dc11548fe2 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:14:12.700449 20139 raft_consensus.cc:482] T 00000000000000000000000000000000 P 25474f26b3ab4143b43529dc11548fe2 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:14:12.700491 20139 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 25474f26b3ab4143b43529dc11548fe2 [term 2 FOLLOWER]: Advancing to term 3
I20260807 15:14:12.701583 20139 raft_consensus.cc:504] T 00000000000000000000000000000000 P 25474f26b3ab4143b43529dc11548fe2 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25474f26b3ab4143b43529dc11548fe2" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 46167 } }
I20260807 15:14:12.701689 20139 leader_election.cc:302] T 00000000000000000000000000000000 P 25474f26b3ab4143b43529dc11548fe2 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 25474f26b3ab4143b43529dc11548fe2; no voters:
I20260807 15:14:12.701870 20139 leader_election.cc:288] T 00000000000000000000000000000000 P 25474f26b3ab4143b43529dc11548fe2 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260807 15:14:12.702000 20146 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 25474f26b3ab4143b43529dc11548fe2 [term 3 FOLLOWER]: Leader election won for term 3
I20260807 15:14:12.702176 20139 sys_catalog.cc:564] T 00000000000000000000000000000000 P 25474f26b3ab4143b43529dc11548fe2 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:14:12.702276 20146 raft_consensus.cc:686] T 00000000000000000000000000000000 P 25474f26b3ab4143b43529dc11548fe2 [term 3 LEADER]: Becoming Leader. State: Replica: 25474f26b3ab4143b43529dc11548fe2, State: Running, Role: LEADER
I20260807 15:14:12.702432 20146 consensus_queue.cc:237] T 00000000000000000000000000000000 P 25474f26b3ab4143b43529dc11548fe2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25474f26b3ab4143b43529dc11548fe2" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 46167 } }
I20260807 15:14:12.702896 20147 sys_catalog.cc:455] T 00000000000000000000000000000000 P 25474f26b3ab4143b43529dc11548fe2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "25474f26b3ab4143b43529dc11548fe2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25474f26b3ab4143b43529dc11548fe2" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 46167 } } }
I20260807 15:14:12.702996 20146 sys_catalog.cc:455] T 00000000000000000000000000000000 P 25474f26b3ab4143b43529dc11548fe2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 25474f26b3ab4143b43529dc11548fe2. Latest consensus state: current_term: 3 leader_uuid: "25474f26b3ab4143b43529dc11548fe2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25474f26b3ab4143b43529dc11548fe2" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 46167 } } }
I20260807 15:14:12.703083 20146 sys_catalog.cc:458] T 00000000000000000000000000000000 P 25474f26b3ab4143b43529dc11548fe2 [sys.catalog]: This master's current role is: LEADER
I20260807 15:14:12.703018 20147 sys_catalog.cc:458] T 00000000000000000000000000000000 P 25474f26b3ab4143b43529dc11548fe2 [sys.catalog]: This master's current role is: LEADER
I20260807 15:14:12.703576 20160 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:14:12.704401 20160 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:14:12.704504 20160 catalog_manager.cc:1259] Loaded cluster ID: 52b11fd4412f4b2f82cffd1af5601a58
I20260807 15:14:12.704537 20160 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260807 15:14:12.704738 20160 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7fa523aa59ad google::LogMessage::Fail() at ??:0
@ 0x7fa523aa888a google::LogMessage::SendToLog() at ??:0
@ 0x7fa523aa54de google::LogMessage::Flush() at ??:0
@ 0x7fa523aa6a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7fa52b9c60f1 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7fa52b9cf8b9 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7fa5259730d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7fa525968954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7fa5267cc6db start_thread at ??:0
@ 0x7fa5225a361f clone at ??:0
W20260807 15:14:12.800102 20144 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:14:12.800376 20144 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:14:12.800416 20144 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:14:12.801735 20144 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:14:12.801786 20144 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:14:12.801815 20144 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.194.1
I20260807 15:14:12.803462 20144 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.194.20:43617
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115480851507-30472-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115480851507-30472-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.1.keytab
--principal=kudu/127.29.194.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.194.1:43899
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115480851507-30472-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.194.1
--webserver_port=43977
--tserver_master_addrs=127.29.194.62:46167
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20144
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.194.1
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115480851507-30472-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:14:12.803718 20144 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:14:12.803977 20144 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 15:14:12.809311 20144 server_base.cc:1034] running on GCE node
W20260807 15:14:12.809290 20193 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:14:12.809405 20202 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:14:12.809290 20195 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:14:12.809808 20144 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:14:12.810060 20144 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:14:12.811369 20144 hybrid_clock.cc:648] HybridClock initialized: now 1786115652811345 us; error 41 us; skew 500 ppm
Aug 07 15:14:12 dist-test-slave-42z9 krb5kdc[18425](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115652, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.194.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:14:12.814816 20144 init.cc:377] Logged in from keytab as kudu/127.29.194.1@KRBTEST.COM (short username kudu)
I20260807 15:14:12.815343 20144 webserver.cc:460] Webserver started at http://127.29.194.1:43977/ using document root <none> and password file <none>
I20260807 15:14:12.815572 20144 fs_manager.cc:359] Metadata directory not provided
I20260807 15:14:12.815624 20144 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:14:12.817027 20144 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:14:12.817804 20214 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:14:12.818020 20144 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:14:12.818150 20144 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115480851507-30472-0/minicluster-data/ts-0/data,/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115480851507-30472-0/minicluster-data/ts-0/wal
uuid: "8dc40eed26fb4a0982ceb06e101d137e"
format_stamp: "Formatted at 2026-08-07 15:14:10 on dist-test-slave-42z9"
server_key: "1df099ade868588b8d17cd7196feb163"
server_key_iv: "04d1e05f9c728096baca1337f00dfa74"
server_key_version: "encryptionkey@0"
I20260807 15:14:12.818521 20144 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115480851507-30472-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115480851507-30472-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115480851507-30472-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:14:12.839362 20144 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:14:12.842068 20144 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:14:12.842221 20144 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:14:12.842481 20144 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:14:12.842953 20222 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260807 15:14:12.845146 20144 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260807 15:14:12.845214 20144 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260807 15:14:12.845265 20144 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260807 15:14:12.846033 20144 ts_tablet_manager.cc:610] Registered 1 tablets
I20260807 15:14:12.846072 20144 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260807 15:14:12.846185 20222 tablet_bootstrap.cc:492] T 492b00874a3a4c3e8be6e36e4789503c P 8dc40eed26fb4a0982ceb06e101d137e: Bootstrap starting.
I20260807 15:14:12.853046 20144 rpc_server.cc:307] RPC server started. Bound to: 127.29.194.1:43899
I20260807 15:14:12.853096 20372 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.194.1:43899 every 8 connection(s)
I20260807 15:14:12.853570 20144 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115480851507-30472-0/minicluster-data/ts-0/data/info.pb
I20260807 15:14:12.857412 20222 log.cc:824] T 492b00874a3a4c3e8be6e36e4789503c P 8dc40eed26fb4a0982ceb06e101d137e: Log is configured to *not* fsync() on all Append() calls
I20260807 15:14:12.860723 20222 tablet_bootstrap.cc:492] T 492b00874a3a4c3e8be6e36e4789503c P 8dc40eed26fb4a0982ceb06e101d137e: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 15:14:12.861097 20222 tablet_bootstrap.cc:492] T 492b00874a3a4c3e8be6e36e4789503c P 8dc40eed26fb4a0982ceb06e101d137e: Bootstrap complete.
I20260807 15:14:12.861379 30472 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7fZSIL/build/release/bin/kudu as pid 20144
I20260807 15:14:12.861665 30472 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7fZSIL/build/release/bin/kudu
/tmp/dist-test-task7fZSIL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115480851507-30472-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115480851507-30472-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115480851507-30472-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115480851507-30472-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.194.2:33663
--local_ip_for_outbound_sockets=127.29.194.2
--tserver_master_addrs=127.29.194.62:46167
--webserver_port=36331
--webserver_interface=127.29.194.2
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.2.keytab
--principal=kudu/127.29.194.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.194.20:43617
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115480851507-30472-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115480851507-30472-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115480851507-30472-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 15:14:12.862349 20222 ts_tablet_manager.cc:1397] T 492b00874a3a4c3e8be6e36e4789503c P 8dc40eed26fb4a0982ceb06e101d137e: Time spent bootstrapping tablet: real 0.016s user 0.000s sys 0.015s
I20260807 15:14:12.863909 20222 raft_consensus.cc:348] T 492b00874a3a4c3e8be6e36e4789503c P 8dc40eed26fb4a0982ceb06e101d137e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff3aa95770c3407f9a7ec35fe5b235c2" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 33663 } } peers { permanent_uuid: "887081dc41e341b7911a7061f5f6d3a1" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 34127 } } peers { permanent_uuid: "8dc40eed26fb4a0982ceb06e101d137e" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 43899 } }
I20260807 15:14:12.864279 20222 raft_consensus.cc:729] T 492b00874a3a4c3e8be6e36e4789503c P 8dc40eed26fb4a0982ceb06e101d137e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8dc40eed26fb4a0982ceb06e101d137e, State: Initialized, Role: FOLLOWER
I20260807 15:14:12.864481 20222 consensus_queue.cc:260] T 492b00874a3a4c3e8be6e36e4789503c P 8dc40eed26fb4a0982ceb06e101d137e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff3aa95770c3407f9a7ec35fe5b235c2" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 33663 } } peers { permanent_uuid: "887081dc41e341b7911a7061f5f6d3a1" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 34127 } } peers { permanent_uuid: "8dc40eed26fb4a0982ceb06e101d137e" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 43899 } }
I20260807 15:14:12.864804 20222 ts_tablet_manager.cc:1428] T 492b00874a3a4c3e8be6e36e4789503c P 8dc40eed26fb4a0982ceb06e101d137e: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
W20260807 15:14:12.913208 20376 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:14:12.854326 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:46167 (local address 127.29.194.1:39165)
0807 15:14:12.854570 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0807 15:14:12.854610 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0807 15:14:12.855098 (+ 488us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:14:12.912977 (+ 57879us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.194.62:46167: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":139,"threads_started":1}
W20260807 15:14:12.913686 20373 heartbeater.cc:644] Failed to heartbeat to 127.29.194.62:46167 (0 consecutive failures): Network error: Failed to ping master at 127.29.194.62:46167: Client connection negotiation failed: client connection to 127.29.194.62:46167: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260807 15:14:12.914202 20376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:12.914020 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:46167 (local address 127.29.194.1:47403)
0807 15:14:12.914092 (+ 72us) negotiation.cc:107] Waiting for socket to connect
0807 15:14:12.914168 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.194.62:46167: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260807 15:14:12.914716 20376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:12.914623 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:46167 (local address 127.29.194.1:57459)
0807 15:14:12.914667 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0807 15:14:12.914696 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.194.62:46167: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260807 15:14:12.914848 20373 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260807 15:14:12.951503 20378 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:14:12.951748 20378 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:14:12.951784 20378 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:14:12.953045 20378 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:14:12.953212 20378 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:14:12.953382 20378 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.194.2
I20260807 15:14:12.956634 20378 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.194.20:43617
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115480851507-30472-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115480851507-30472-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.2.keytab
--principal=kudu/127.29.194.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.194.2:33663
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115480851507-30472-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.194.2
--webserver_port=36331
--tserver_master_addrs=127.29.194.62:46167
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20378
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.194.2
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115480851507-30472-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:14:12.957008 20378 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:14:12.957283 20378 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:14:12.962954 20385 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:14:12.963114 20387 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:14:12.963196 20378 server_base.cc:1034] running on GCE node
W20260807 15:14:12.962954 20389 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:14:12.963608 20378 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:14:12.963924 20378 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:14:12.965149 20378 hybrid_clock.cc:648] HybridClock initialized: now 1786115652965111 us; error 61 us; skew 500 ppm
Aug 07 15:14:12 dist-test-slave-42z9 krb5kdc[18425](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115652, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.194.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:14:12.968400 20378 init.cc:377] Logged in from keytab as kudu/127.29.194.2@KRBTEST.COM (short username kudu)
I20260807 15:14:12.968870 20378 webserver.cc:460] Webserver started at http://127.29.194.2:36331/ using document root <none> and password file <none>
I20260807 15:14:12.969087 20378 fs_manager.cc:359] Metadata directory not provided
I20260807 15:14:12.969130 20378 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:14:12.970577 20378 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:14:12.971194 20397 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:14:12.971390 20378 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:14:12.971481 20378 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115480851507-30472-0/minicluster-data/ts-1/data,/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115480851507-30472-0/minicluster-data/ts-1/wal
uuid: "ff3aa95770c3407f9a7ec35fe5b235c2"
format_stamp: "Formatted at 2026-08-07 15:14:11 on dist-test-slave-42z9"
server_key: "2138424ff93118e26d34ccb996845cb7"
server_key_iv: "7ea05b46921a36dc1c9400216469249f"
server_key_version: "encryptionkey@0"
I20260807 15:14:12.971861 20378 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115480851507-30472-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115480851507-30472-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115480851507-30472-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:14:12.982682 20378 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:14:12.984831 20378 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:14:12.984978 20378 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:14:12.985212 20378 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:14:12.985592 20378 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:14:12.985630 20378 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:14:12.985661 20378 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:14:12.985682 20378 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:14:12.991819 20378 rpc_server.cc:307] RPC server started. Bound to: 127.29.194.2:33663
I20260807 15:14:12.991876 20556 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.194.2:33663 every 8 connection(s)
I20260807 15:14:12.992250 20378 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115480851507-30472-0/minicluster-data/ts-1/data/info.pb
I20260807 15:14:12.993500 20564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:12.992966 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:46167 (local address 127.29.194.2:50793)
0807 15:14:12.993178 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0807 15:14:12.993269 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.194.62:46167: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":89,"threads_started":1}
W20260807 15:14:12.993924 20557 heartbeater.cc:644] Failed to heartbeat to 127.29.194.62:46167 (0 consecutive failures): Network error: Failed to ping master at 127.29.194.62:46167: Client connection negotiation failed: client connection to 127.29.194.62:46167: connect: Connection refused (error 111)
I20260807 15:14:12.994259 20564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:12.994161 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:46167 (local address 127.29.194.2:55067)
0807 15:14:12.994210 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0807 15:14:12.994239 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.194.62:46167: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260807 15:14:12.994860 20564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:12.994718 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:46167 (local address 127.29.194.2:43885)
0807 15:14:12.994804 (+ 86us) negotiation.cc:107] Waiting for socket to connect
0807 15:14:12.994838 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.194.62:46167: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
W20260807 15:14:12.995050 20557 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260807 15:14:13.000638 30472 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7fZSIL/build/release/bin/kudu as pid 20378
I20260807 15:14:13.000897 30472 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7fZSIL/build/release/bin/kudu
/tmp/dist-test-task7fZSIL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115480851507-30472-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115480851507-30472-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115480851507-30472-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115480851507-30472-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.194.3:34127
--local_ip_for_outbound_sockets=127.29.194.3
--tserver_master_addrs=127.29.194.62:46167
--webserver_port=32813
--webserver_interface=127.29.194.3
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.3.keytab
--principal=kudu/127.29.194.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.194.20:43617
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115480851507-30472-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115480851507-30472-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115480851507-30472-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:14:13.090425 20565 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:14:13.090667 20565 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:14:13.090695 20565 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:14:13.092306 20565 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:14:13.092455 20565 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:14:13.092504 20565 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.194.3
I20260807 15:14:13.095512 20565 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.194.20:43617
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115480851507-30472-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115480851507-30472-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.3.keytab
--principal=kudu/127.29.194.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.194.3:34127
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115480851507-30472-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.194.3
--webserver_port=32813
--tserver_master_addrs=127.29.194.62:46167
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20565
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.194.3
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115480851507-30472-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:14:13.095925 20565 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:14:13.096236 20565 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:14:13.103343 20579 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:14:13.103513 20565 server_base.cc:1034] running on GCE node
W20260807 15:14:13.103361 20575 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:14:13.103361 20572 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:14:13.103969 20565 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:14:13.104310 20565 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:14:13.105561 20565 hybrid_clock.cc:648] HybridClock initialized: now 1786115653105463 us; error 119 us; skew 500 ppm
Aug 07 15:14:13 dist-test-slave-42z9 krb5kdc[18425](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115653, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.194.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:14:13.108340 20565 init.cc:377] Logged in from keytab as kudu/127.29.194.3@KRBTEST.COM (short username kudu)
I20260807 15:14:13.108915 20565 webserver.cc:460] Webserver started at http://127.29.194.3:32813/ using document root <none> and password file <none>
I20260807 15:14:13.109163 20565 fs_manager.cc:359] Metadata directory not provided
I20260807 15:14:13.109217 20565 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:14:13.110785 20565 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:14:13.111829 20587 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:14:13.112087 20565 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:14:13.112205 20565 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115480851507-30472-0/minicluster-data/ts-2/data,/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115480851507-30472-0/minicluster-data/ts-2/wal
uuid: "887081dc41e341b7911a7061f5f6d3a1"
format_stamp: "Formatted at 2026-08-07 15:14:11 on dist-test-slave-42z9"
server_key: "2447b01858ab937b1461a913272b57a5"
server_key_iv: "b75c3f8a4ad4573456a01f06f0fa49a4"
server_key_version: "encryptionkey@0"
I20260807 15:14:13.112620 20565 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115480851507-30472-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115480851507-30472-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115480851507-30472-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:14:13.120666 20565 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:14:13.123000 20565 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:14:13.123149 20565 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:14:13.123384 20565 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:14:13.123687 20565 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:14:13.123720 20565 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:14:13.123750 20565 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:14:13.123782 20565 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:14:13.130184 20565 rpc_server.cc:307] RPC server started. Bound to: 127.29.194.3:34127
I20260807 15:14:13.130224 20753 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.194.3:34127 every 8 connection(s)
I20260807 15:14:13.130595 20565 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115480851507-30472-0/minicluster-data/ts-2/data/info.pb
I20260807 15:14:13.132120 20757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:13.131380 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:46167 (local address 127.29.194.3:39009)
0807 15:14:13.131728 (+ 348us) negotiation.cc:107] Waiting for socket to connect
0807 15:14:13.131888 (+ 160us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.194.62:46167: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":227,"thread_start_us":92,"threads_started":1}
W20260807 15:14:13.132637 20755 heartbeater.cc:644] Failed to heartbeat to 127.29.194.62:46167 (0 consecutive failures): Network error: Failed to ping master at 127.29.194.62:46167: Client connection negotiation failed: client connection to 127.29.194.62:46167: connect: Connection refused (error 111)
I20260807 15:14:13.132987 20757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:13.132855 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:46167 (local address 127.29.194.3:39367)
0807 15:14:13.132913 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0807 15:14:13.132954 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.194.62:46167: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260807 15:14:13.133303 20757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:13.133222 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:46167 (local address 127.29.194.3:40161)
0807 15:14:13.133262 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0807 15:14:13.133286 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.194.62:46167: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
W20260807 15:14:13.133405 20755 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260807 15:14:13.137607 30472 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7fZSIL/build/release/bin/kudu as pid 20565
I20260807 15:14:13.915591 20759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:13.915314 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:46167 (local address 127.29.194.1:34035)
0807 15:14:13.915468 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0807 15:14:13.915517 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.194.62:46167: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":73,"threads_started":1}
I20260807 15:14:13.995647 20760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:13.995408 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:46167 (local address 127.29.194.2:48187)
0807 15:14:13.995543 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0807 15:14:13.995583 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.194.62:46167: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":62,"threads_started":1}
I20260807 15:14:14.134011 20761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:14.133785 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:46167 (local address 127.29.194.3:42329)
0807 15:14:14.133920 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0807 15:14:14.133953 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.194.62:46167: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":70,"threads_started":1}
I20260807 15:14:14.727234 20763 raft_consensus.cc:482] T 492b00874a3a4c3e8be6e36e4789503c P 8dc40eed26fb4a0982ceb06e101d137e [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 15:14:14.727370 20763 raft_consensus.cc:504] T 492b00874a3a4c3e8be6e36e4789503c P 8dc40eed26fb4a0982ceb06e101d137e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff3aa95770c3407f9a7ec35fe5b235c2" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 33663 } } peers { permanent_uuid: "887081dc41e341b7911a7061f5f6d3a1" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 34127 } } peers { permanent_uuid: "8dc40eed26fb4a0982ceb06e101d137e" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 43899 } }
I20260807 15:14:14.727771 20763 leader_election.cc:288] T 492b00874a3a4c3e8be6e36e4789503c P 8dc40eed26fb4a0982ceb06e101d137e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ff3aa95770c3407f9a7ec35fe5b235c2 (127.29.194.2:33663), 887081dc41e341b7911a7061f5f6d3a1 (127.29.194.3:34127)
Aug 07 15:14:14 dist-test-slave-42z9 krb5kdc[18425](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115652, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.194.1@KRBTEST.COM for kudu/127.29.194.2@KRBTEST.COM
Aug 07 15:14:14 dist-test-slave-42z9 krb5kdc[18425](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115652, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.194.1@KRBTEST.COM for kudu/127.29.194.3@KRBTEST.COM
I20260807 15:14:14.741078 20771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:14.728171 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.194.1:57847 (local address 127.29.194.3:34127)
0807 15:14:14.728298 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0807 15:14:14.728307 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 15:14:14.728325 (+ 18us) server_negotiation.cc:407] Connection header received
0807 15:14:14.728739 (+ 414us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:14:14.728741 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:14:14.728784 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:14:14.728916 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:14:14.729880 (+ 964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:14.731375 (+ 1495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:14.733141 (+ 1766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:14.733770 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:14.733854 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:14.737373 (+ 3519us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:14:14.737376 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:14:14.737377 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:14:14.737389 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:14:14.739208 (+ 1819us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:14:14.739703 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:14:14.739706 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:14:14.739707 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:14:14.739838 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:14:14.740137 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:14:14.740138 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:14:14.740139 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:14:14.740475 (+ 336us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:14:14.740552 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0807 15:14:14.740836 (+ 284us) server_negotiation.cc:299] Negotiation successful
0807 15:14:14.740959 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":97,"thread_start_us":55,"threads_started":1}
I20260807 15:14:14.741068 20764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:14.727949 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.3:34127 (local address 127.29.194.1:57847)
0807 15:14:14.728079 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0807 15:14:14.728091 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 15:14:14.728151 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:14:14.728983 (+ 832us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:14:14.728986 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:14:14.729391 (+ 405us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:14:14.729718 (+ 327us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:14:14.729726 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:14:14.731510 (+ 1784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:14:14.731511 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:14:14.733078 (+ 1567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:14:14.733081 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:14:14.733835 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:14:14.733836 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:14:14.733901 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:14.735234 (+ 1333us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:14:14.735239 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:14:14.737278 (+ 2039us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:14:14.739274 (+ 1996us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:14:14.739275 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:14:14.739275 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:14:14.739616 (+ 341us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:14:14.739899 (+ 283us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:14:14.739899 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:14:14.739900 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:14:14.740063 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:14:14.740551 (+ 488us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:14:14.740559 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:14:14.740684 (+ 125us) client_negotiation.cc:769] Sending connection context
0807 15:14:14.740774 (+ 90us) client_negotiation.cc:240] Negotiation successful
0807 15:14:14.740960 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"mutex_wait_us":838,"thread_start_us":60,"threads_started":1}
I20260807 15:14:14.741400 20768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:14.727951 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.2:33663 (local address 127.29.194.1:44379)
0807 15:14:14.728153 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0807 15:14:14.728162 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0807 15:14:14.728205 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:14:14.728994 (+ 789us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:14:14.728995 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:14:14.729395 (+ 400us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:14:14.729718 (+ 323us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:14:14.729726 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:14:14.731525 (+ 1799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:14:14.731526 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:14:14.732760 (+ 1234us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:14:14.732765 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:14:14.733729 (+ 964us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:14:14.733731 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:14:14.733866 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:14.734718 (+ 852us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:14:14.734729 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:14:14.736795 (+ 2066us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:14:14.739127 (+ 2332us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:14:14.739137 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:14:14.739140 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:14:14.739616 (+ 476us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:14:14.739915 (+ 299us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:14:14.739916 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:14:14.739916 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:14:14.740065 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:14:14.741173 (+ 1108us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:14:14.741174 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:14:14.741215 (+ 41us) client_negotiation.cc:769] Sending connection context
0807 15:14:14.741269 (+ 54us) client_negotiation.cc:240] Negotiation successful
0807 15:14:14.741341 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":189,"mutex_wait_us":80,"thread_start_us":51,"threads_started":1}
I20260807 15:14:14.742144 20770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:14.728089 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.194.1:44379 (local address 127.29.194.2:33663)
0807 15:14:14.728253 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0807 15:14:14.728266 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 15:14:14.728290 (+ 24us) server_negotiation.cc:407] Connection header received
0807 15:14:14.728748 (+ 458us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:14:14.728750 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:14:14.728793 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:14:14.728933 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:14:14.729880 (+ 947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:14.731376 (+ 1496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:14.732870 (+ 1494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:14.733553 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:14.733675 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:14.736912 (+ 3237us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:14:14.736917 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:14:14.736917 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:14:14.736930 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:14:14.739045 (+ 2115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:14:14.739722 (+ 677us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:14:14.739725 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:14:14.739726 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:14:14.739841 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:14:14.740137 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:14:14.740138 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:14:14.740138 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:14:14.740474 (+ 336us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:14:14.740560 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0807 15:14:14.741890 (+ 1330us) server_negotiation.cc:299] Negotiation successful
0807 15:14:14.742042 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129,"thread_start_us":57,"threads_started":1}
W20260807 15:14:14.742415 20217 leader_election.cc:341] T 492b00874a3a4c3e8be6e36e4789503c P 8dc40eed26fb4a0982ceb06e101d137e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 887081dc41e341b7911a7061f5f6d3a1 (127.29.194.3:34127): Not found: Tablet not found: 492b00874a3a4c3e8be6e36e4789503c
W20260807 15:14:14.742874 20218 leader_election.cc:341] T 492b00874a3a4c3e8be6e36e4789503c P 8dc40eed26fb4a0982ceb06e101d137e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ff3aa95770c3407f9a7ec35fe5b235c2 (127.29.194.2:33663): Not found: Tablet not found: 492b00874a3a4c3e8be6e36e4789503c
I20260807 15:14:14.742923 20218 leader_election.cc:302] T 492b00874a3a4c3e8be6e36e4789503c P 8dc40eed26fb4a0982ceb06e101d137e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8dc40eed26fb4a0982ceb06e101d137e; no voters: 887081dc41e341b7911a7061f5f6d3a1, ff3aa95770c3407f9a7ec35fe5b235c2
I20260807 15:14:14.743088 20763 raft_consensus.cc:2726] T 492b00874a3a4c3e8be6e36e4789503c P 8dc40eed26fb4a0982ceb06e101d137e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 15:14:14.916456 20768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:14.916307 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:46167 (local address 127.29.194.1:34309)
0807 15:14:14.916360 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0807 15:14:14.916405 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.194.62:46167: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
I20260807 15:14:14.996599 20772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:14.996204 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:46167 (local address 127.29.194.2:60387)
0807 15:14:14.996418 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0807 15:14:14.996516 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.194.62:46167: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":67,"threads_started":1}
I20260807 15:14:15.134810 20774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:15.134525 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:46167 (local address 127.29.194.3:48939)
0807 15:14:15.134698 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0807 15:14:15.134753 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.194.62:46167: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":73,"threads_started":1}
I20260807 15:14:15.917294 20775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:15.916959 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:46167 (local address 127.29.194.1:37131)
0807 15:14:15.917174 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0807 15:14:15.917226 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.194.62:46167: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":78,"threads_started":1}
I20260807 15:14:15.997625 20776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:15.997276 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:46167 (local address 127.29.194.2:34413)
0807 15:14:15.997512 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0807 15:14:15.997573 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.194.62:46167: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":73,"threads_started":1}
I20260807 15:14:16.135836 20777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:16.135482 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:46167 (local address 127.29.194.3:56265)
0807 15:14:16.135716 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0807 15:14:16.135782 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.194.62:46167: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":129,"threads_started":1}
I20260807 15:14:16.405278 20778 raft_consensus.cc:482] T 492b00874a3a4c3e8be6e36e4789503c P 8dc40eed26fb4a0982ceb06e101d137e [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 15:14:16.405398 20778 raft_consensus.cc:504] T 492b00874a3a4c3e8be6e36e4789503c P 8dc40eed26fb4a0982ceb06e101d137e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff3aa95770c3407f9a7ec35fe5b235c2" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 33663 } } peers { permanent_uuid: "887081dc41e341b7911a7061f5f6d3a1" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 34127 } } peers { permanent_uuid: "8dc40eed26fb4a0982ceb06e101d137e" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 43899 } }
I20260807 15:14:16.405645 20778 leader_election.cc:288] T 492b00874a3a4c3e8be6e36e4789503c P 8dc40eed26fb4a0982ceb06e101d137e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ff3aa95770c3407f9a7ec35fe5b235c2 (127.29.194.2:33663), 887081dc41e341b7911a7061f5f6d3a1 (127.29.194.3:34127)
W20260807 15:14:16.406085 20218 leader_election.cc:341] T 492b00874a3a4c3e8be6e36e4789503c P 8dc40eed26fb4a0982ceb06e101d137e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ff3aa95770c3407f9a7ec35fe5b235c2 (127.29.194.2:33663): Not found: Tablet not found: 492b00874a3a4c3e8be6e36e4789503c
W20260807 15:14:16.406169 20217 leader_election.cc:341] T 492b00874a3a4c3e8be6e36e4789503c P 8dc40eed26fb4a0982ceb06e101d137e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 887081dc41e341b7911a7061f5f6d3a1 (127.29.194.3:34127): Not found: Tablet not found: 492b00874a3a4c3e8be6e36e4789503c
I20260807 15:14:16.406258 20217 leader_election.cc:302] T 492b00874a3a4c3e8be6e36e4789503c P 8dc40eed26fb4a0982ceb06e101d137e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8dc40eed26fb4a0982ceb06e101d137e; no voters: 887081dc41e341b7911a7061f5f6d3a1, ff3aa95770c3407f9a7ec35fe5b235c2
I20260807 15:14:16.406347 20778 raft_consensus.cc:2726] T 492b00874a3a4c3e8be6e36e4789503c P 8dc40eed26fb4a0982ceb06e101d137e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 15:14:16.918321 20779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:16.917950 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:46167 (local address 127.29.194.1:37025)
0807 15:14:16.918218 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0807 15:14:16.918269 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.194.62:46167: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":85,"threads_started":1}
I20260807 15:14:16.998651 20780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:16.998287 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:46167 (local address 127.29.194.2:48455)
0807 15:14:16.998536 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0807 15:14:16.998599 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.194.62:46167: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":221,"thread_start_us":122,"threads_started":1}
I20260807 15:14:17.136938 20781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:17.136596 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:46167 (local address 127.29.194.3:48121)
0807 15:14:17.136789 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0807 15:14:17.136872 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.194.62:46167: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":77,"threads_started":1}
I20260807 15:14:17.919582 20782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:17.918987 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:46167 (local address 127.29.194.1:47515)
0807 15:14:17.919482 (+ 495us) negotiation.cc:107] Waiting for socket to connect
0807 15:14:17.919534 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.194.62:46167: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":469,"thread_start_us":96,"threads_started":1}
I20260807 15:14:17.988834 20785 raft_consensus.cc:482] T 492b00874a3a4c3e8be6e36e4789503c P 8dc40eed26fb4a0982ceb06e101d137e [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 15:14:17.988942 20785 raft_consensus.cc:504] T 492b00874a3a4c3e8be6e36e4789503c P 8dc40eed26fb4a0982ceb06e101d137e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff3aa95770c3407f9a7ec35fe5b235c2" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 33663 } } peers { permanent_uuid: "887081dc41e341b7911a7061f5f6d3a1" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 34127 } } peers { permanent_uuid: "8dc40eed26fb4a0982ceb06e101d137e" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 43899 } }
I20260807 15:14:17.989120 20785 leader_election.cc:288] T 492b00874a3a4c3e8be6e36e4789503c P 8dc40eed26fb4a0982ceb06e101d137e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ff3aa95770c3407f9a7ec35fe5b235c2 (127.29.194.2:33663), 887081dc41e341b7911a7061f5f6d3a1 (127.29.194.3:34127)
W20260807 15:14:17.989566 20217 leader_election.cc:341] T 492b00874a3a4c3e8be6e36e4789503c P 8dc40eed26fb4a0982ceb06e101d137e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 887081dc41e341b7911a7061f5f6d3a1 (127.29.194.3:34127): Not found: Tablet not found: 492b00874a3a4c3e8be6e36e4789503c
W20260807 15:14:17.989632 20218 leader_election.cc:341] T 492b00874a3a4c3e8be6e36e4789503c P 8dc40eed26fb4a0982ceb06e101d137e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ff3aa95770c3407f9a7ec35fe5b235c2 (127.29.194.2:33663): Not found: Tablet not found: 492b00874a3a4c3e8be6e36e4789503c
I20260807 15:14:17.989704 20218 leader_election.cc:302] T 492b00874a3a4c3e8be6e36e4789503c P 8dc40eed26fb4a0982ceb06e101d137e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8dc40eed26fb4a0982ceb06e101d137e; no voters: 887081dc41e341b7911a7061f5f6d3a1, ff3aa95770c3407f9a7ec35fe5b235c2
I20260807 15:14:17.989771 20785 raft_consensus.cc:2726] T 492b00874a3a4c3e8be6e36e4789503c P 8dc40eed26fb4a0982ceb06e101d137e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 15:14:17.999524 20786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:17.999221 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:46167 (local address 127.29.194.2:34193)
0807 15:14:17.999410 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0807 15:14:17.999454 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.194.62:46167: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":77,"threads_started":1}
I20260807 15:14:18.138041 20787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:18.137672 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:46167 (local address 127.29.194.3:46901)
0807 15:14:18.137950 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0807 15:14:18.137992 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.194.62:46167: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":256,"thread_start_us":99,"threads_started":1}
I20260807 15:14:18.920819 20788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:18.920497 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:46167 (local address 127.29.194.1:51227)
0807 15:14:18.920703 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0807 15:14:18.920767 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.194.62:46167: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":100,"threads_started":1}
I20260807 15:14:19.000365 20789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:19.000085 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:46167 (local address 127.29.194.2:40631)
0807 15:14:19.000261 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0807 15:14:19.000300 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.194.62:46167: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":92,"threads_started":1}
I20260807 15:14:19.138988 20790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:19.138713 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:46167 (local address 127.29.194.3:32831)
0807 15:14:19.138881 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0807 15:14:19.138940 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.194.62:46167: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":71,"threads_started":1}
I20260807 15:14:19.921831 20791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:19.921534 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:46167 (local address 127.29.194.1:57807)
0807 15:14:19.921729 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0807 15:14:19.921777 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.194.62:46167: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":77,"threads_started":1}
I20260807 15:14:20.001353 20792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:20.000883 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:46167 (local address 127.29.194.2:51421)
0807 15:14:20.001149 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0807 15:14:20.001292 (+ 143us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.194.62:46167: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":234,"thread_start_us":70,"threads_started":1}
I20260807 15:14:20.139776 20793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:20.139524 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:46167 (local address 127.29.194.3:52135)
0807 15:14:20.139686 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0807 15:14:20.139726 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.194.62:46167: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":79,"threads_started":1}
I20260807 15:14:20.922621 20795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:20.922388 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:46167 (local address 127.29.194.1:37383)
0807 15:14:20.922535 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0807 15:14:20.922572 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.194.62:46167: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":71,"threads_started":1}
I20260807 15:14:21.002187 20799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:21.001947 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:46167 (local address 127.29.194.2:48523)
0807 15:14:21.002090 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0807 15:14:21.002136 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.194.62:46167: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":62,"threads_started":1}
I20260807 15:14:21.140615 20802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:21.140359 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:46167 (local address 127.29.194.3:50147)
0807 15:14:21.140519 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0807 15:14:21.140566 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.194.62:46167: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":68,"threads_started":1}
I20260807 15:14:21.163851 20803 raft_consensus.cc:482] T 492b00874a3a4c3e8be6e36e4789503c P 8dc40eed26fb4a0982ceb06e101d137e [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 15:14:21.163961 20803 raft_consensus.cc:504] T 492b00874a3a4c3e8be6e36e4789503c P 8dc40eed26fb4a0982ceb06e101d137e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff3aa95770c3407f9a7ec35fe5b235c2" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 33663 } } peers { permanent_uuid: "887081dc41e341b7911a7061f5f6d3a1" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 34127 } } peers { permanent_uuid: "8dc40eed26fb4a0982ceb06e101d137e" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 43899 } }
I20260807 15:14:21.164115 20803 leader_election.cc:288] T 492b00874a3a4c3e8be6e36e4789503c P 8dc40eed26fb4a0982ceb06e101d137e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ff3aa95770c3407f9a7ec35fe5b235c2 (127.29.194.2:33663), 887081dc41e341b7911a7061f5f6d3a1 (127.29.194.3:34127)
W20260807 15:14:21.164523 20218 leader_election.cc:341] T 492b00874a3a4c3e8be6e36e4789503c P 8dc40eed26fb4a0982ceb06e101d137e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ff3aa95770c3407f9a7ec35fe5b235c2 (127.29.194.2:33663): Not found: Tablet not found: 492b00874a3a4c3e8be6e36e4789503c
W20260807 15:14:21.164606 20217 leader_election.cc:341] T 492b00874a3a4c3e8be6e36e4789503c P 8dc40eed26fb4a0982ceb06e101d137e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 887081dc41e341b7911a7061f5f6d3a1 (127.29.194.3:34127): Not found: Tablet not found: 492b00874a3a4c3e8be6e36e4789503c
I20260807 15:14:21.164649 20217 leader_election.cc:302] T 492b00874a3a4c3e8be6e36e4789503c P 8dc40eed26fb4a0982ceb06e101d137e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8dc40eed26fb4a0982ceb06e101d137e; no voters: 887081dc41e341b7911a7061f5f6d3a1, ff3aa95770c3407f9a7ec35fe5b235c2
I20260807 15:14:21.164716 20803 raft_consensus.cc:2726] T 492b00874a3a4c3e8be6e36e4789503c P 8dc40eed26fb4a0982ceb06e101d137e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 15:14:21.923250 20804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:21.923052 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:46167 (local address 127.29.194.1:50095)
0807 15:14:21.923170 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0807 15:14:21.923202 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.194.62:46167: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":69,"threads_started":1}
I20260807 15:14:22.002946 20805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:22.002701 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:46167 (local address 127.29.194.2:42087)
0807 15:14:22.002850 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0807 15:14:22.002897 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.194.62:46167: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":68,"threads_started":1}
I20260807 15:14:22.141507 20806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:22.141202 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:46167 (local address 127.29.194.3:40519)
0807 15:14:22.141372 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0807 15:14:22.141419 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.194.62:46167: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":67,"threads_started":1}
I20260807 15:14:22.923949 20807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:22.923713 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:46167 (local address 127.29.194.1:55297)
0807 15:14:22.923856 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0807 15:14:22.923895 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.194.62:46167: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":65,"threads_started":1}
I20260807 15:14:23.003844 20808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:23.003538 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:46167 (local address 127.29.194.2:60923)
0807 15:14:23.003743 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0807 15:14:23.003794 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.194.62:46167: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":75,"threads_started":1}
I20260807 15:14:23.142365 20809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:23.142073 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:46167 (local address 127.29.194.3:55965)
0807 15:14:23.142270 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0807 15:14:23.142317 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.194.62:46167: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":69,"threads_started":1}
I20260807 15:14:23.924826 20810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:23.924557 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:46167 (local address 127.29.194.1:54785)
0807 15:14:23.924722 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0807 15:14:23.924771 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.194.62:46167: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":73,"threads_started":1}
I20260807 15:14:24.004756 20813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:24.004455 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:46167 (local address 127.29.194.2:35521)
0807 15:14:24.004655 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0807 15:14:24.004706 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.194.62:46167: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":106,"threads_started":1}
I20260807 15:14:24.143189 20816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:24.142950 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:46167 (local address 127.29.194.3:41139)
0807 15:14:24.143104 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0807 15:14:24.143141 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.194.62:46167: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":93,"threads_started":1}
I20260807 15:14:24.207924 20817 raft_consensus.cc:482] T 492b00874a3a4c3e8be6e36e4789503c P 8dc40eed26fb4a0982ceb06e101d137e [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 15:14:24.208041 20817 raft_consensus.cc:504] T 492b00874a3a4c3e8be6e36e4789503c P 8dc40eed26fb4a0982ceb06e101d137e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff3aa95770c3407f9a7ec35fe5b235c2" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 33663 } } peers { permanent_uuid: "887081dc41e341b7911a7061f5f6d3a1" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 34127 } } peers { permanent_uuid: "8dc40eed26fb4a0982ceb06e101d137e" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 43899 } }
I20260807 15:14:24.208225 20817 leader_election.cc:288] T 492b00874a3a4c3e8be6e36e4789503c P 8dc40eed26fb4a0982ceb06e101d137e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ff3aa95770c3407f9a7ec35fe5b235c2 (127.29.194.2:33663), 887081dc41e341b7911a7061f5f6d3a1 (127.29.194.3:34127)
W20260807 15:14:24.208592 20218 leader_election.cc:341] T 492b00874a3a4c3e8be6e36e4789503c P 8dc40eed26fb4a0982ceb06e101d137e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ff3aa95770c3407f9a7ec35fe5b235c2 (127.29.194.2:33663): Not found: Tablet not found: 492b00874a3a4c3e8be6e36e4789503c
W20260807 15:14:24.208674 20217 leader_election.cc:341] T 492b00874a3a4c3e8be6e36e4789503c P 8dc40eed26fb4a0982ceb06e101d137e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 887081dc41e341b7911a7061f5f6d3a1 (127.29.194.3:34127): Not found: Tablet not found: 492b00874a3a4c3e8be6e36e4789503c
I20260807 15:14:24.208752 20217 leader_election.cc:302] T 492b00874a3a4c3e8be6e36e4789503c P 8dc40eed26fb4a0982ceb06e101d137e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8dc40eed26fb4a0982ceb06e101d137e; no voters: 887081dc41e341b7911a7061f5f6d3a1, ff3aa95770c3407f9a7ec35fe5b235c2
I20260807 15:14:24.208835 20817 raft_consensus.cc:2726] T 492b00874a3a4c3e8be6e36e4789503c P 8dc40eed26fb4a0982ceb06e101d137e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 15:14:24.925697 20818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:24.925426 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:46167 (local address 127.29.194.1:57957)
0807 15:14:24.925602 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0807 15:14:24.925648 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.194.62:46167: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":112,"threads_started":1}
I20260807 15:14:25.005887 20819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:25.005609 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:46167 (local address 127.29.194.2:35177)
0807 15:14:25.005791 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0807 15:14:25.005837 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.194.62:46167: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":88,"threads_started":1}
I20260807 15:14:25.144071 20820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:25.143761 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:46167 (local address 127.29.194.3:38433)
0807 15:14:25.143963 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0807 15:14:25.144020 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.194.62:46167: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":65,"threads_started":1}
I20260807 15:14:25.926406 20821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:25.926199 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:46167 (local address 127.29.194.1:47457)
0807 15:14:25.926320 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0807 15:14:25.926358 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.194.62:46167: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":66,"threads_started":1}
I20260807 15:14:26.006711 20822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:26.006459 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:46167 (local address 127.29.194.2:33623)
0807 15:14:26.006609 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0807 15:14:26.006658 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.194.62:46167: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":64,"threads_started":1}
I20260807 15:14:26.144934 20823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:26.144674 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:46167 (local address 127.29.194.3:59457)
0807 15:14:26.144842 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0807 15:14:26.144887 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.194.62:46167: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":78,"threads_started":1}
I20260807 15:14:26.927184 20824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:26.926895 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:46167 (local address 127.29.194.1:58749)
0807 15:14:26.927079 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0807 15:14:26.927128 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.194.62:46167: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":98,"threads_started":1}
I20260807 15:14:27.007591 20825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:27.007336 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:46167 (local address 127.29.194.2:43435)
0807 15:14:27.007492 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0807 15:14:27.007535 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.194.62:46167: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":68,"threads_started":1}
I20260807 15:14:27.145776 20826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:27.145531 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:46167 (local address 127.29.194.3:40809)
0807 15:14:27.145680 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0807 15:14:27.145726 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.194.62:46167: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":83,"threads_started":1}
I20260807 15:14:27.928160 20828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:27.927893 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:46167 (local address 127.29.194.1:40873)
0807 15:14:27.928067 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0807 15:14:27.928114 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.194.62:46167: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":86,"threads_started":1}
I20260807 15:14:28.008546 20830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:28.008272 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:46167 (local address 127.29.194.2:36865)
0807 15:14:28.008446 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0807 15:14:28.008491 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.194.62:46167: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":97,"threads_started":1}
I20260807 15:14:28.146700 20831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:28.146425 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:46167 (local address 127.29.194.3:40377)
0807 15:14:28.146608 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0807 15:14:28.146653 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.194.62:46167: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":95,"threads_started":1}
I20260807 15:14:28.147183 30472 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7fZSIL/build/release/bin/kudu with pid 20144
I20260807 15:14:28.160375 30472 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7fZSIL/build/release/bin/kudu with pid 20378
I20260807 15:14:28.171936 30472 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7fZSIL/build/release/bin/kudu with pid 20565
2026-08-07T15:14:28Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (18881 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115480851507-30472-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 15:14:28 dist-test-slave-42z9 krb5kdc[20839](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 15:14:28 dist-test-slave-42z9 krb5kdc[20839](info): set up 2 sockets
Aug 07 15:14:28 dist-test-slave-42z9 krb5kdc[20839](info): commencing operation
krb5kdc: starting...
W20260807 15:14:29.353008 30472 mini_kdc.cc:121] Time spent starting KDC: real 1.150s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115480851507-30472-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115480851507-30472-0/krb5kdc/test-user.keytab.
Aug 07 15:14:29 dist-test-slave-42z9 krb5kdc[20839](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115669, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T15:14:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T15:14:29Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.194.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.194.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.194.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.62.keytab.
Entry for principal kudu/127.29.194.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.62.keytab.
I20260807 15:14:29.480757 30472 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7fZSIL/build/release/bin/kudu
/tmp/dist-test-task7fZSIL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115480851507-30472-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115480851507-30472-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115480851507-30472-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115480851507-30472-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.194.62:42465
--webserver_interface=127.29.194.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.62.keytab
--principal=kudu/127.29.194.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.194.20:40443
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.194.62:42465
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115480851507-30472-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115480851507-30472-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115480851507-30472-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:14:29.568717 20860 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:14:29.568956 20860 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:14:29.568980 20860 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:14:29.570237 20860 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260807 15:14:29.570289 20860 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260807 15:14:29.570302 20860 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260807 15:14:29.570313 20860 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:14:29.570325 20860 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:14:29.570336 20860 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:14:29.570348 20860 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:14:29.570359 20860 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:14:29.571971 20860 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.194.20:40443
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115480851507-30472-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115480851507-30472-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.29.194.62:42465
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.62.keytab
--principal=kudu/127.29.194.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.194.62:42465
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115480851507-30472-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.194.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20860
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115480851507-30472-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:14:29.572211 20860 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:14:29.572423 20860 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:14:29.578377 20868 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:14:29.578511 20867 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:14:29.578385 20860 server_base.cc:1034] running on GCE node
W20260807 15:14:29.578377 20871 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:14:29.578873 20860 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:14:29.579313 20860 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:14:29.580559 20860 hybrid_clock.cc:648] HybridClock initialized: now 1786115669580521 us; error 38 us; skew 500 ppm
Aug 07 15:14:29 dist-test-slave-42z9 krb5kdc[20839](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115669, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.194.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:14:29.583968 20860 init.cc:377] Logged in from keytab as kudu/127.29.194.62@KRBTEST.COM (short username kudu)
I20260807 15:14:29.584599 20860 webserver.cc:460] Webserver started at http://127.29.194.62:41213/ using document root <none> and password file <none>
I20260807 15:14:29.584848 20860 fs_manager.cc:359] Metadata directory not provided
I20260807 15:14:29.584903 20860 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:14:29.585047 20860 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:14:29.586534 20860 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115480851507-30472-0/minicluster-data/master-0/data/instance:
uuid: "4e95b6a1e91e4fc0b95dbb205a0d0d4a"
format_stamp: "Formatted at 2026-08-07 15:14:29 on dist-test-slave-42z9"
server_key: "964c398495f7750df66021b9acd44de6"
server_key_iv: "79b9c8fbbc418736323465fe6474d8fc"
server_key_version: "encryptionkey@0"
I20260807 15:14:29.587270 20860 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115480851507-30472-0/minicluster-data/master-0/wal/instance:
uuid: "4e95b6a1e91e4fc0b95dbb205a0d0d4a"
format_stamp: "Formatted at 2026-08-07 15:14:29 on dist-test-slave-42z9"
server_key: "964c398495f7750df66021b9acd44de6"
server_key_iv: "79b9c8fbbc418736323465fe6474d8fc"
server_key_version: "encryptionkey@0"
I20260807 15:14:29.588979 20860 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.001s
I20260807 15:14:29.589921 20881 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:14:29.590131 20860 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260807 15:14:29.590241 20860 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115480851507-30472-0/minicluster-data/master-0/data,/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115480851507-30472-0/minicluster-data/master-0/wal
uuid: "4e95b6a1e91e4fc0b95dbb205a0d0d4a"
format_stamp: "Formatted at 2026-08-07 15:14:29 on dist-test-slave-42z9"
server_key: "964c398495f7750df66021b9acd44de6"
server_key_iv: "79b9c8fbbc418736323465fe6474d8fc"
server_key_version: "encryptionkey@0"
I20260807 15:14:29.590323 20860 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115480851507-30472-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115480851507-30472-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115480851507-30472-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:14:29.612728 20860 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:14:29.615020 20860 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:14:29.615171 20860 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:14:29.619294 20860 rpc_server.cc:307] RPC server started. Bound to: 127.29.194.62:42465
I20260807 15:14:29.619359 20960 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.194.62:42465 every 8 connection(s)
I20260807 15:14:29.619736 20860 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115480851507-30472-0/minicluster-data/master-0/data/info.pb
I20260807 15:14:29.620685 20961 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:14:29.623200 20961 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4e95b6a1e91e4fc0b95dbb205a0d0d4a: Bootstrap starting.
I20260807 15:14:29.623924 20961 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4e95b6a1e91e4fc0b95dbb205a0d0d4a: Neither blocks nor log segments found. Creating new log.
I20260807 15:14:29.624225 20961 log.cc:824] T 00000000000000000000000000000000 P 4e95b6a1e91e4fc0b95dbb205a0d0d4a: Log is configured to *not* fsync() on all Append() calls
I20260807 15:14:29.625428 20961 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4e95b6a1e91e4fc0b95dbb205a0d0d4a: No bootstrap required, opened a new log
I20260807 15:14:29.627123 30472 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7fZSIL/build/release/bin/kudu as pid 20860
I20260807 15:14:29.627231 30472 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115480851507-30472-0/minicluster-data/master-0/wal/instance
I20260807 15:14:29.627413 30472 external_mini_cluster.cc:1428] Setting key bc6613aebfdd5f27dc4a0b9386fe67cc
I20260807 15:14:29.628198 20961 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4e95b6a1e91e4fc0b95dbb205a0d0d4a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e95b6a1e91e4fc0b95dbb205a0d0d4a" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 42465 } }
I20260807 15:14:29.628383 20961 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4e95b6a1e91e4fc0b95dbb205a0d0d4a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:14:29.628417 20961 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4e95b6a1e91e4fc0b95dbb205a0d0d4a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e95b6a1e91e4fc0b95dbb205a0d0d4a, State: Initialized, Role: FOLLOWER
I20260807 15:14:29.628571 20961 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4e95b6a1e91e4fc0b95dbb205a0d0d4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e95b6a1e91e4fc0b95dbb205a0d0d4a" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 42465 } }
I20260807 15:14:29.628651 20961 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4e95b6a1e91e4fc0b95dbb205a0d0d4a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:14:29.628684 20961 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4e95b6a1e91e4fc0b95dbb205a0d0d4a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:14:29.628736 20961 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4e95b6a1e91e4fc0b95dbb205a0d0d4a [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:14:29.629827 20961 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4e95b6a1e91e4fc0b95dbb205a0d0d4a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e95b6a1e91e4fc0b95dbb205a0d0d4a" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 42465 } }
I20260807 15:14:29.629931 20961 leader_election.cc:302] T 00000000000000000000000000000000 P 4e95b6a1e91e4fc0b95dbb205a0d0d4a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4e95b6a1e91e4fc0b95dbb205a0d0d4a; no voters:
I20260807 15:14:29.630100 20961 leader_election.cc:288] T 00000000000000000000000000000000 P 4e95b6a1e91e4fc0b95dbb205a0d0d4a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 15:14:29.630159 20968 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4e95b6a1e91e4fc0b95dbb205a0d0d4a [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:14:29.630362 20961 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4e95b6a1e91e4fc0b95dbb205a0d0d4a [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:14:29.630386 20968 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4e95b6a1e91e4fc0b95dbb205a0d0d4a [term 1 LEADER]: Becoming Leader. State: Replica: 4e95b6a1e91e4fc0b95dbb205a0d0d4a, State: Running, Role: LEADER
I20260807 15:14:29.630493 20968 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4e95b6a1e91e4fc0b95dbb205a0d0d4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e95b6a1e91e4fc0b95dbb205a0d0d4a" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 42465 } }
I20260807 15:14:29.630834 20968 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4e95b6a1e91e4fc0b95dbb205a0d0d4a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4e95b6a1e91e4fc0b95dbb205a0d0d4a. Latest consensus state: current_term: 1 leader_uuid: "4e95b6a1e91e4fc0b95dbb205a0d0d4a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e95b6a1e91e4fc0b95dbb205a0d0d4a" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 42465 } } }
I20260807 15:14:29.630975 20968 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4e95b6a1e91e4fc0b95dbb205a0d0d4a [sys.catalog]: This master's current role is: LEADER
I20260807 15:14:29.631217 20977 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:14:29.631735 20969 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4e95b6a1e91e4fc0b95dbb205a0d0d4a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4e95b6a1e91e4fc0b95dbb205a0d0d4a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e95b6a1e91e4fc0b95dbb205a0d0d4a" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 42465 } } }
I20260807 15:14:29.631814 20969 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4e95b6a1e91e4fc0b95dbb205a0d0d4a [sys.catalog]: This master's current role is: LEADER
I20260807 15:14:29.631949 20977 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:14:29.633572 20977 catalog_manager.cc:1347] Generated new cluster ID: e054a19df1ae4e99a1ba8f154ecc4884
I20260807 15:14:29.633630 20977 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 07 15:14:29 dist-test-slave-42z9 krb5kdc[20839](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115669, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.194.62@KRBTEST.COM
I20260807 15:14:29.639330 20967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:29.628014 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34466 (local address 127.29.194.62:42465)
0807 15:14:29.628313 (+ 299us) server_negotiation.cc:206] Beginning negotiation
0807 15:14:29.628326 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 15:14:29.628370 (+ 44us) server_negotiation.cc:407] Connection header received
0807 15:14:29.628549 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:14:29.628558 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:14:29.628637 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:14:29.628843 (+ 206us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:14:29.629803 (+ 960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:29.631235 (+ 1432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:29.632292 (+ 1057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:29.632865 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:29.632996 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:29.635671 (+ 2675us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:14:29.635675 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:14:29.635676 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:14:29.635688 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:14:29.637694 (+ 2006us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:14:29.638063 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:14:29.638067 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:14:29.638067 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:14:29.638210 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:14:29.638387 (+ 177us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:14:29.638388 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:14:29.638389 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:14:29.638711 (+ 322us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:14:29.638860 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0807 15:14:29.638968 (+ 108us) server_negotiation.cc:299] Negotiation successful
0807 15:14:29.639119 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":225,"thread_start_us":129,"threads_started":1}
I20260807 15:14:29.647199 20977 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:14:29.647719 20977 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:14:29.650951 20977 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4e95b6a1e91e4fc0b95dbb205a0d0d4a: Generated new TSK 0
I20260807 15:14:29.651124 20977 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.29.194.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.194.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.194.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.1.keytab.
Entry for principal kudu/127.29.194.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.1.keytab.
I20260807 15:14:29.682407 30472 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7fZSIL/build/release/bin/kudu
/tmp/dist-test-task7fZSIL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115480851507-30472-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115480851507-30472-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115480851507-30472-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115480851507-30472-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.194.1:0
--local_ip_for_outbound_sockets=127.29.194.1
--webserver_interface=127.29.194.1
--webserver_port=0
--tserver_master_addrs=127.29.194.62:42465
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.1.keytab
--principal=kudu/127.29.194.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.194.20:40443
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115480851507-30472-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115480851507-30472-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115480851507-30472-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:14:29.768020 20994 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:14:29.768256 20994 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:14:29.768292 20994 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:14:29.769556 20994 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:14:29.769610 20994 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:14:29.769649 20994 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.194.1
I20260807 15:14:29.771272 20994 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.194.20:40443
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115480851507-30472-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115480851507-30472-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.1.keytab
--principal=kudu/127.29.194.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.194.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115480851507-30472-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.194.1
--webserver_port=0
--tserver_master_addrs=127.29.194.62:42465
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20994
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.194.1
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115480851507-30472-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:14:29.771552 20994 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:14:29.771787 20994 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:14:29.777215 21006 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:14:29.777387 21003 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:14:29.777387 20994 server_base.cc:1034] running on GCE node
W20260807 15:14:29.777536 21004 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:14:29.777913 20994 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:14:29.778306 20994 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:14:29.779510 20994 hybrid_clock.cc:648] HybridClock initialized: now 1786115669779479 us; error 48 us; skew 500 ppm
Aug 07 15:14:29 dist-test-slave-42z9 krb5kdc[20839](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115669, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.194.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:14:29.782477 20994 init.cc:377] Logged in from keytab as kudu/127.29.194.1@KRBTEST.COM (short username kudu)
I20260807 15:14:29.782996 20994 webserver.cc:460] Webserver started at http://127.29.194.1:40433/ using document root <none> and password file <none>
I20260807 15:14:29.783236 20994 fs_manager.cc:359] Metadata directory not provided
I20260807 15:14:29.783290 20994 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:14:29.783428 20994 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:14:29.784543 20994 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115480851507-30472-0/minicluster-data/ts-0/data/instance:
uuid: "246195cca7b043878aaaf04013697524"
format_stamp: "Formatted at 2026-08-07 15:14:29 on dist-test-slave-42z9"
server_key: "11e8656c00110a93fcf480a485a0826c"
server_key_iv: "4716fd9bfb72a0e363b1a5b3a1dc54f1"
server_key_version: "encryptionkey@0"
I20260807 15:14:29.784904 20994 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115480851507-30472-0/minicluster-data/ts-0/wal/instance:
uuid: "246195cca7b043878aaaf04013697524"
format_stamp: "Formatted at 2026-08-07 15:14:29 on dist-test-slave-42z9"
server_key: "11e8656c00110a93fcf480a485a0826c"
server_key_iv: "4716fd9bfb72a0e363b1a5b3a1dc54f1"
server_key_version: "encryptionkey@0"
I20260807 15:14:29.786464 20994 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260807 15:14:29.787575 21013 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:14:29.787786 20994 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 15:14:29.787884 20994 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115480851507-30472-0/minicluster-data/ts-0/data,/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115480851507-30472-0/minicluster-data/ts-0/wal
uuid: "246195cca7b043878aaaf04013697524"
format_stamp: "Formatted at 2026-08-07 15:14:29 on dist-test-slave-42z9"
server_key: "11e8656c00110a93fcf480a485a0826c"
server_key_iv: "4716fd9bfb72a0e363b1a5b3a1dc54f1"
server_key_version: "encryptionkey@0"
I20260807 15:14:29.787958 20994 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115480851507-30472-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115480851507-30472-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115480851507-30472-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:14:29.816345 20994 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:14:29.819530 20994 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:14:29.819702 20994 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:14:29.819953 20994 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:14:29.820274 20994 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:14:29.820308 20994 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:14:29.820338 20994 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:14:29.820364 20994 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:14:29.826557 20994 rpc_server.cc:307] RPC server started. Bound to: 127.29.194.1:41283
I20260807 15:14:29.826613 21176 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.194.1:41283 every 8 connection(s)
I20260807 15:14:29.826963 20994 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115480851507-30472-0/minicluster-data/ts-0/data/info.pb
I20260807 15:14:29.829103 30472 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7fZSIL/build/release/bin/kudu as pid 20994
I20260807 15:14:29.829203 30472 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115480851507-30472-0/minicluster-data/ts-0/wal/instance
I20260807 15:14:29.829389 30472 external_mini_cluster.cc:1428] Setting key 3bc24f462a3b20b9d6deaa8eaf8aa846
Aug 07 15:14:29 dist-test-slave-42z9 krb5kdc[20839](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115669, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.194.1@KRBTEST.COM for kudu/127.29.194.62@KRBTEST.COM
I20260807 15:14:29.837867 20967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:29.827818 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.194.1:46427 (local address 127.29.194.62:42465)
0807 15:14:29.827887 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0807 15:14:29.827890 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:14:29.828080 (+ 190us) server_negotiation.cc:407] Connection header received
0807 15:14:29.828597 (+ 517us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:14:29.828598 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:14:29.828613 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:14:29.828672 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:14:29.829822 (+ 1150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:29.830528 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:29.832248 (+ 1720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:29.832482 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:29.832542 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:29.835183 (+ 2641us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:14:29.835185 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:14:29.835186 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:14:29.835198 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:14:29.836500 (+ 1302us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:14:29.836894 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:14:29.836895 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:14:29.836896 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:14:29.836950 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:14:29.837175 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:14:29.837176 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:14:29.837177 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:14:29.837291 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:14:29.837348 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0807 15:14:29.837683 (+ 335us) server_negotiation.cc:299] Negotiation successful
0807 15:14:29.837775 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260807 15:14:29.838111 21181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:29.827782 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:42465 (local address 127.29.194.1:46427)
0807 15:14:29.827952 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0807 15:14:29.827991 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0807 15:14:29.828478 (+ 487us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:14:29.828740 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:14:29.828744 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:14:29.829407 (+ 663us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:14:29.829747 (+ 340us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:14:29.829753 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:14:29.830678 (+ 925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:14:29.830679 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:14:29.832148 (+ 1469us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:14:29.832151 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:14:29.832570 (+ 419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:14:29.832571 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:14:29.832720 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:29.833276 (+ 556us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:14:29.833285 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:14:29.835077 (+ 1792us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:14:29.836584 (+ 1507us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:14:29.836592 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:14:29.836595 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:14:29.836824 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:14:29.837008 (+ 184us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:14:29.837009 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:14:29.837009 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:14:29.837118 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:14:29.837348 (+ 230us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:14:29.837356 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:14:29.837556 (+ 200us) client_negotiation.cc:769] Sending connection context
0807 15:14:29.837653 (+ 97us) client_negotiation.cc:240] Negotiation successful
0807 15:14:29.837828 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":88,"threads_started":1}
I20260807 15:14:29.838599 21177 heartbeater.cc:347] Connected to a master server at 127.29.194.62:42465
I20260807 15:14:29.838716 21177 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.29.194.2@KRBTEST.COM; defaulting to no policy
I20260807 15:14:29.838932 21177 heartbeater.cc:511] Master 127.29.194.62:42465 requested a full tablet report, sending...
I20260807 15:14:29.839516 20909 ts_manager.cc:194] Registered new tserver with Master: 246195cca7b043878aaaf04013697524 (127.29.194.1:41283)
I20260807 15:14:29.840200 20909 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.194.1@KRBTEST.COM'} at 127.29.194.1:46427
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.194.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.194.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.2.keytab.
Entry for principal kudu/127.29.194.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.2.keytab.
I20260807 15:14:29.861204 30472 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7fZSIL/build/release/bin/kudu
/tmp/dist-test-task7fZSIL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115480851507-30472-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115480851507-30472-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115480851507-30472-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115480851507-30472-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.194.2:0
--local_ip_for_outbound_sockets=127.29.194.2
--webserver_interface=127.29.194.2
--webserver_port=0
--tserver_master_addrs=127.29.194.62:42465
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.2.keytab
--principal=kudu/127.29.194.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.194.20:40443
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115480851507-30472-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115480851507-30472-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115480851507-30472-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:14:29.945617 21184 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:14:29.945850 21184 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:14:29.945876 21184 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:14:29.947077 21184 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:14:29.947127 21184 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:14:29.947154 21184 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.194.2
I20260807 15:14:29.948704 21184 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.194.20:40443
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115480851507-30472-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115480851507-30472-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.2.keytab
--principal=kudu/127.29.194.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.194.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115480851507-30472-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.194.2
--webserver_port=0
--tserver_master_addrs=127.29.194.62:42465
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21184
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.194.2
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115480851507-30472-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:14:29.948947 21184 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:14:29.949162 21184 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:14:29.954598 21195 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:14:29.954648 21184 server_base.cc:1034] running on GCE node
W20260807 15:14:29.954599 21190 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:14:29.954598 21192 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:14:29.955339 21184 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:14:29.955641 21184 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:14:29.956847 21184 hybrid_clock.cc:648] HybridClock initialized: now 1786115669956832 us; error 30 us; skew 500 ppm
Aug 07 15:14:29 dist-test-slave-42z9 krb5kdc[20839](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115669, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.194.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:14:29.959851 21184 init.cc:377] Logged in from keytab as kudu/127.29.194.2@KRBTEST.COM (short username kudu)
I20260807 15:14:29.960363 21184 webserver.cc:460] Webserver started at http://127.29.194.2:36107/ using document root <none> and password file <none>
I20260807 15:14:29.960667 21184 fs_manager.cc:359] Metadata directory not provided
I20260807 15:14:29.960728 21184 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:14:29.960844 21184 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:14:29.961953 21184 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115480851507-30472-0/minicluster-data/ts-1/data/instance:
uuid: "732c0e3bda8644778e6306e1aa2eb6d0"
format_stamp: "Formatted at 2026-08-07 15:14:29 on dist-test-slave-42z9"
server_key: "4127a96f8d5a6dfc46b331de9e58b49f"
server_key_iv: "c45eaa31b7403e481f8cbc847079cf50"
server_key_version: "encryptionkey@0"
I20260807 15:14:29.962296 21184 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115480851507-30472-0/minicluster-data/ts-1/wal/instance:
uuid: "732c0e3bda8644778e6306e1aa2eb6d0"
format_stamp: "Formatted at 2026-08-07 15:14:29 on dist-test-slave-42z9"
server_key: "4127a96f8d5a6dfc46b331de9e58b49f"
server_key_iv: "c45eaa31b7403e481f8cbc847079cf50"
server_key_version: "encryptionkey@0"
I20260807 15:14:29.963685 21184 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 15:14:29.964474 21210 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:14:29.964720 21184 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:14:29.964813 21184 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115480851507-30472-0/minicluster-data/ts-1/data,/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115480851507-30472-0/minicluster-data/ts-1/wal
uuid: "732c0e3bda8644778e6306e1aa2eb6d0"
format_stamp: "Formatted at 2026-08-07 15:14:29 on dist-test-slave-42z9"
server_key: "4127a96f8d5a6dfc46b331de9e58b49f"
server_key_iv: "c45eaa31b7403e481f8cbc847079cf50"
server_key_version: "encryptionkey@0"
I20260807 15:14:29.964892 21184 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115480851507-30472-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115480851507-30472-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115480851507-30472-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:14:29.974253 21184 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:14:29.976109 21184 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:14:29.976250 21184 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:14:29.976469 21184 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:14:29.976802 21184 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:14:29.976845 21184 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:14:29.976876 21184 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:14:29.976892 21184 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:14:29.983505 21184 rpc_server.cc:307] RPC server started. Bound to: 127.29.194.2:37205
I20260807 15:14:29.983566 21379 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.194.2:37205 every 8 connection(s)
I20260807 15:14:29.983887 21184 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115480851507-30472-0/minicluster-data/ts-1/data/info.pb
I20260807 15:14:29.987607 30472 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7fZSIL/build/release/bin/kudu as pid 21184
I20260807 15:14:29.987867 30472 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115480851507-30472-0/minicluster-data/ts-1/wal/instance
I20260807 15:14:29.988134 30472 external_mini_cluster.cc:1428] Setting key 6b0d8345a77047d66c991bf4b4729eb5
Aug 07 15:14:29 dist-test-slave-42z9 krb5kdc[20839](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115669, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.194.2@KRBTEST.COM for kudu/127.29.194.62@KRBTEST.COM
I20260807 15:14:29.994378 20967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:29.984716 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.194.2:57215 (local address 127.29.194.62:42465)
0807 15:14:29.984802 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0807 15:14:29.984804 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:14:29.984929 (+ 125us) server_negotiation.cc:407] Connection header received
0807 15:14:29.985497 (+ 568us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:14:29.985499 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:14:29.985514 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:14:29.985581 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:14:29.986587 (+ 1006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:29.987114 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:29.988910 (+ 1796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:29.989110 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:29.989167 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:29.991666 (+ 2499us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:14:29.991668 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:14:29.991668 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:14:29.991676 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:14:29.992988 (+ 1312us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:14:29.993393 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:14:29.993395 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:14:29.993396 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:14:29.993473 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:14:29.993725 (+ 252us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:14:29.993726 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:14:29.993726 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:14:29.993846 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:14:29.993906 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0807 15:14:29.994186 (+ 280us) server_negotiation.cc:299] Negotiation successful
0807 15:14:29.994283 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260807 15:14:29.994592 21387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:29.984620 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:42465 (local address 127.29.194.2:57215)
0807 15:14:29.984796 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0807 15:14:29.984835 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0807 15:14:29.985355 (+ 520us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:14:29.985640 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:14:29.985643 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:14:29.986242 (+ 599us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:14:29.986445 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:14:29.986450 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:14:29.987341 (+ 891us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:14:29.987344 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:14:29.988817 (+ 1473us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:14:29.988823 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:14:29.989200 (+ 377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:14:29.989201 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:14:29.989346 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:29.989914 (+ 568us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:14:29.989924 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:14:29.991558 (+ 1634us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:14:29.993071 (+ 1513us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:14:29.993080 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:14:29.993083 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:14:29.993323 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:14:29.993533 (+ 210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:14:29.993534 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:14:29.993535 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:14:29.993651 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:14:29.993904 (+ 253us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:14:29.993912 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:14:29.994064 (+ 152us) client_negotiation.cc:769] Sending connection context
0807 15:14:29.994160 (+ 96us) client_negotiation.cc:240] Negotiation successful
0807 15:14:29.994347 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":75,"threads_started":1}
I20260807 15:14:29.995107 21381 heartbeater.cc:347] Connected to a master server at 127.29.194.62:42465
I20260807 15:14:29.995234 21381 heartbeater.cc:464] Registering TS with master...
I20260807 15:14:29.995440 21381 heartbeater.cc:511] Master 127.29.194.62:42465 requested a full tablet report, sending...
I20260807 15:14:29.995898 20909 ts_manager.cc:194] Registered new tserver with Master: 732c0e3bda8644778e6306e1aa2eb6d0 (127.29.194.2:37205)
I20260807 15:14:29.996315 20909 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.194.2@KRBTEST.COM'} at 127.29.194.2:57215
WARNING: no policy specified for kudu/127.29.194.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.194.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.194.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.3.keytab.
Entry for principal kudu/127.29.194.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.3.keytab.
I20260807 15:14:30.018577 30472 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7fZSIL/build/release/bin/kudu
/tmp/dist-test-task7fZSIL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115480851507-30472-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115480851507-30472-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115480851507-30472-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115480851507-30472-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.194.3:0
--local_ip_for_outbound_sockets=127.29.194.3
--webserver_interface=127.29.194.3
--webserver_port=0
--tserver_master_addrs=127.29.194.62:42465
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.3.keytab
--principal=kudu/127.29.194.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.194.20:40443
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115480851507-30472-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115480851507-30472-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115480851507-30472-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:14:30.107010 21390 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:14:30.107244 21390 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:14:30.107270 21390 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:14:30.108439 21390 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:14:30.108484 21390 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:14:30.108510 21390 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.194.3
I20260807 15:14:30.110033 21390 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.194.20:40443
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115480851507-30472-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115480851507-30472-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.3.keytab
--principal=kudu/127.29.194.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.194.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115480851507-30472-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.194.3
--webserver_port=0
--tserver_master_addrs=127.29.194.62:42465
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21390
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.194.3
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115480851507-30472-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:14:30.110265 21390 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:14:30.110482 21390 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:14:30.115760 21396 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:14:30.115749 21400 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:14:30.115792 21390 server_base.cc:1034] running on GCE node
W20260807 15:14:30.115754 21397 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:14:30.116200 21390 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:14:30.116473 21390 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:14:30.117671 21390 hybrid_clock.cc:648] HybridClock initialized: now 1786115670117657 us; error 28 us; skew 500 ppm
Aug 07 15:14:30 dist-test-slave-42z9 krb5kdc[20839](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115670, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.194.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:14:30.120332 21390 init.cc:377] Logged in from keytab as kudu/127.29.194.3@KRBTEST.COM (short username kudu)
I20260807 15:14:30.120824 21390 webserver.cc:460] Webserver started at http://127.29.194.3:40145/ using document root <none> and password file <none>
I20260807 15:14:30.121060 21390 fs_manager.cc:359] Metadata directory not provided
I20260807 15:14:30.121112 21390 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:14:30.121240 21390 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:14:30.122376 21390 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115480851507-30472-0/minicluster-data/ts-2/data/instance:
uuid: "d76b0cdebcc44295a6d2a7049ef4b030"
format_stamp: "Formatted at 2026-08-07 15:14:30 on dist-test-slave-42z9"
server_key: "e0006eb0c6c241ecdc04fe98b9047bc8"
server_key_iv: "45699283dc90bfe884ea33d509e9b9b3"
server_key_version: "encryptionkey@0"
I20260807 15:14:30.122746 21390 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115480851507-30472-0/minicluster-data/ts-2/wal/instance:
uuid: "d76b0cdebcc44295a6d2a7049ef4b030"
format_stamp: "Formatted at 2026-08-07 15:14:30 on dist-test-slave-42z9"
server_key: "e0006eb0c6c241ecdc04fe98b9047bc8"
server_key_iv: "45699283dc90bfe884ea33d509e9b9b3"
server_key_version: "encryptionkey@0"
I20260807 15:14:30.124117 21390 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260807 15:14:30.124953 21410 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:14:30.125241 21390 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 15:14:30.125340 21390 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115480851507-30472-0/minicluster-data/ts-2/data,/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115480851507-30472-0/minicluster-data/ts-2/wal
uuid: "d76b0cdebcc44295a6d2a7049ef4b030"
format_stamp: "Formatted at 2026-08-07 15:14:30 on dist-test-slave-42z9"
server_key: "e0006eb0c6c241ecdc04fe98b9047bc8"
server_key_iv: "45699283dc90bfe884ea33d509e9b9b3"
server_key_version: "encryptionkey@0"
I20260807 15:14:30.125428 21390 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115480851507-30472-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115480851507-30472-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115480851507-30472-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:14:30.137970 21390 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:14:30.140347 21390 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:14:30.140496 21390 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:14:30.140758 21390 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:14:30.141080 21390 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:14:30.141117 21390 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:14:30.141147 21390 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:14:30.141171 21390 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:14:30.147058 21390 rpc_server.cc:307] RPC server started. Bound to: 127.29.194.3:34775
I20260807 15:14:30.147111 21562 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.194.3:34775 every 8 connection(s)
I20260807 15:14:30.147426 21390 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115480851507-30472-0/minicluster-data/ts-2/data/info.pb
Aug 07 15:14:30 dist-test-slave-42z9 krb5kdc[20839](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115670, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.194.3@KRBTEST.COM for kudu/127.29.194.62@KRBTEST.COM
I20260807 15:14:30.155748 30472 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7fZSIL/build/release/bin/kudu as pid 21390
I20260807 15:14:30.155853 30472 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115480851507-30472-0/minicluster-data/ts-2/wal/instance
I20260807 15:14:30.156059 30472 external_mini_cluster.cc:1428] Setting key ca2a449aece86bc6f62ed4b2932e51e2
I20260807 15:14:30.157097 20967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:30.148169 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.194.3:55269 (local address 127.29.194.62:42465)
0807 15:14:30.148246 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0807 15:14:30.148248 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:14:30.148394 (+ 146us) server_negotiation.cc:407] Connection header received
0807 15:14:30.148906 (+ 512us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:14:30.148907 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:14:30.148918 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:14:30.148958 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:14:30.149936 (+ 978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:30.150480 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:30.151621 (+ 1141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:30.151811 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:30.151872 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:30.154080 (+ 2208us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:14:30.154081 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:14:30.154081 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:14:30.154089 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:14:30.155504 (+ 1415us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:14:30.156021 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:14:30.156022 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:14:30.156023 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:14:30.156084 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:14:30.156420 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:14:30.156421 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:14:30.156421 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:14:30.156573 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:14:30.156639 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0807 15:14:30.156946 (+ 307us) server_negotiation.cc:299] Negotiation successful
0807 15:14:30.157033 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260807 15:14:30.157300 21566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:30.148118 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:42465 (local address 127.29.194.3:55269)
0807 15:14:30.148275 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0807 15:14:30.148317 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0807 15:14:30.148791 (+ 474us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:14:30.149023 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:14:30.149027 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:14:30.149645 (+ 618us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:14:30.149848 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:14:30.149853 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:14:30.150591 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:14:30.150592 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:14:30.151548 (+ 956us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:14:30.151551 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:14:30.151881 (+ 330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:14:30.151882 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:14:30.151980 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:30.152480 (+ 500us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:14:30.152494 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:14:30.153974 (+ 1480us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:14:30.155591 (+ 1617us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:14:30.155602 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:14:30.155604 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:14:30.155931 (+ 327us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:14:30.156202 (+ 271us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:14:30.156203 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:14:30.156203 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:14:30.156345 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:14:30.156640 (+ 295us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:14:30.156646 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:14:30.156808 (+ 162us) client_negotiation.cc:769] Sending connection context
0807 15:14:30.156916 (+ 108us) client_negotiation.cc:240] Negotiation successful
0807 15:14:30.157088 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":71,"threads_started":1}
I20260807 15:14:30.157773 21563 heartbeater.cc:347] Connected to a master server at 127.29.194.62:42465
I20260807 15:14:30.157876 21563 heartbeater.cc:464] Registering TS with master...
I20260807 15:14:30.158056 21563 heartbeater.cc:511] Master 127.29.194.62:42465 requested a full tablet report, sending...
I20260807 15:14:30.158413 20909 ts_manager.cc:194] Registered new tserver with Master: d76b0cdebcc44295a6d2a7049ef4b030 (127.29.194.3:34775)
I20260807 15:14:30.158749 20909 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.194.3@KRBTEST.COM'} at 127.29.194.3:55269
I20260807 15:14:30.168056 30472 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 15:14:30.175585 20967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:30.169538 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41860 (local address 127.29.194.62:42465)
0807 15:14:30.169602 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0807 15:14:30.169604 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:14:30.169686 (+ 82us) server_negotiation.cc:407] Connection header received
0807 15:14:30.169734 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:14:30.169734 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:14:30.169743 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:14:30.169791 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:14:30.170316 (+ 525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:30.170815 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:30.171540 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:30.171725 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:30.171782 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:30.173024 (+ 1242us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:14:30.173025 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:14:30.173025 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:14:30.173032 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:14:30.174346 (+ 1314us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:14:30.174726 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:14:30.174726 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:14:30.174727 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:14:30.174776 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:14:30.175023 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:14:30.175024 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:14:30.175024 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:14:30.175198 (+ 174us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:14:30.175300 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0807 15:14:30.175449 (+ 149us) server_negotiation.cc:299] Negotiation successful
0807 15:14:30.175518 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260807 15:14:30.177731 20909 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41860:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 15:14:30.179363 20909 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 15:14:30.186149 21581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:30.181877 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.3:34775 (local address 127.0.0.1:44646)
0807 15:14:30.182276 (+ 399us) negotiation.cc:107] Waiting for socket to connect
0807 15:14:30.182285 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0807 15:14:30.182380 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:14:30.182636 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:14:30.182640 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:14:30.182643 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:14:30.182717 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:14:30.182720 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:14:30.183664 (+ 944us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:14:30.183665 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:14:30.184845 (+ 1180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:14:30.184848 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:14:30.185790 (+ 942us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:14:30.185791 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:14:30.185926 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:30.185941 (+ 15us) client_negotiation.cc:769] Sending connection context
0807 15:14:30.186012 (+ 71us) client_negotiation.cc:240] Negotiation successful
0807 15:14:30.186067 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":377,"thread_start_us":62,"threads_started":1}
I20260807 15:14:30.186169 21583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:30.182016 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44646 (local address 127.29.194.3:34775)
0807 15:14:30.182221 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0807 15:14:30.182225 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:14:30.182351 (+ 126us) server_negotiation.cc:407] Connection header received
0807 15:14:30.182472 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:14:30.182475 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:14:30.182490 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:14:30.182556 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:14:30.182792 (+ 236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:30.183567 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:30.184957 (+ 1390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:30.185692 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:30.185766 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:30.185825 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0807 15:14:30.186043 (+ 218us) server_negotiation.cc:299] Negotiation successful
0807 15:14:30.186090 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":172,"thread_start_us":103,"threads_started":1}
I20260807 15:14:30.186451 21579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:30.181688 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.1:41283 (local address 127.0.0.1:52316)
0807 15:14:30.182002 (+ 314us) negotiation.cc:107] Waiting for socket to connect
0807 15:14:30.182042 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0807 15:14:30.182119 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:14:30.182692 (+ 573us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:14:30.182693 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:14:30.182695 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:14:30.182744 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:14:30.182746 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:14:30.184093 (+ 1347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:14:30.184094 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:14:30.185098 (+ 1004us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:14:30.185100 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:14:30.186134 (+ 1034us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:14:30.186135 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:14:30.186199 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:30.186203 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 15:14:30.186337 (+ 134us) client_negotiation.cc:240] Negotiation successful
0807 15:14:30.186385 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":286,"thread_start_us":46,"threads_started":1}
I20260807 15:14:30.186519 21580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:30.181803 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52316 (local address 127.29.194.1:41283)
0807 15:14:30.181990 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0807 15:14:30.181995 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:14:30.182117 (+ 122us) server_negotiation.cc:407] Connection header received
0807 15:14:30.182213 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:14:30.182215 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:14:30.182232 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:14:30.182335 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:14:30.182845 (+ 510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:30.183981 (+ 1136us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:30.185198 (+ 1217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:30.185976 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:30.186044 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:30.186098 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0807 15:14:30.186375 (+ 277us) server_negotiation.cc:299] Negotiation successful
0807 15:14:30.186422 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":150,"thread_start_us":90,"threads_started":1}
I20260807 15:14:30.187338 21473 tablet_service.cc:1467] Processing CreateTablet for tablet f6dc8f3e065e427086299e5878b671b5 (DEFAULT_TABLE table=test-table [id=115c5af53e0c426fbac39efc831e81f8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:14:30.187569 21083 tablet_service.cc:1467] Processing CreateTablet for tablet f6dc8f3e065e427086299e5878b671b5 (DEFAULT_TABLE table=test-table [id=115c5af53e0c426fbac39efc831e81f8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:14:30.187677 21473 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f6dc8f3e065e427086299e5878b671b5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:14:30.187855 21083 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f6dc8f3e065e427086299e5878b671b5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:14:30.189824 21582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:30.182024 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.2:37205 (local address 127.0.0.1:57186)
0807 15:14:30.182519 (+ 495us) negotiation.cc:107] Waiting for socket to connect
0807 15:14:30.182529 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 15:14:30.182584 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:14:30.182813 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:14:30.182814 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:14:30.182816 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:14:30.182852 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:14:30.182853 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:14:30.185179 (+ 2326us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:14:30.185179 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:14:30.186013 (+ 834us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:14:30.186015 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:14:30.189572 (+ 3557us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:14:30.189573 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:14:30.189647 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:30.189652 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 15:14:30.189706 (+ 54us) client_negotiation.cc:240] Negotiation successful
0807 15:14:30.189752 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":474,"thread_start_us":69,"threads_started":1}
I20260807 15:14:30.190124 21584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:30.182231 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57186 (local address 127.29.194.2:37205)
0807 15:14:30.182532 (+ 301us) server_negotiation.cc:206] Beginning negotiation
0807 15:14:30.182536 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:14:30.182578 (+ 42us) server_negotiation.cc:407] Connection header received
0807 15:14:30.182649 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:14:30.182651 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:14:30.182666 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:14:30.182737 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:14:30.182924 (+ 187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:30.184001 (+ 1077us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:30.186095 (+ 2094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:30.186809 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:30.186873 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:30.186925 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0807 15:14:30.189993 (+ 3068us) server_negotiation.cc:299] Negotiation successful
0807 15:14:30.190048 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":264,"thread_start_us":193,"threads_started":1}
I20260807 15:14:30.190606 21585 tablet_bootstrap.cc:492] T f6dc8f3e065e427086299e5878b671b5 P d76b0cdebcc44295a6d2a7049ef4b030: Bootstrap starting.
I20260807 15:14:30.190783 21276 tablet_service.cc:1467] Processing CreateTablet for tablet f6dc8f3e065e427086299e5878b671b5 (DEFAULT_TABLE table=test-table [id=115c5af53e0c426fbac39efc831e81f8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:14:30.191079 21276 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f6dc8f3e065e427086299e5878b671b5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:14:30.191274 21585 tablet_bootstrap.cc:654] T f6dc8f3e065e427086299e5878b671b5 P d76b0cdebcc44295a6d2a7049ef4b030: Neither blocks nor log segments found. Creating new log.
I20260807 15:14:30.191743 21585 log.cc:824] T f6dc8f3e065e427086299e5878b671b5 P d76b0cdebcc44295a6d2a7049ef4b030: Log is configured to *not* fsync() on all Append() calls
I20260807 15:14:30.192126 21586 tablet_bootstrap.cc:492] T f6dc8f3e065e427086299e5878b671b5 P 246195cca7b043878aaaf04013697524: Bootstrap starting.
I20260807 15:14:30.192565 21585 tablet_bootstrap.cc:492] T f6dc8f3e065e427086299e5878b671b5 P d76b0cdebcc44295a6d2a7049ef4b030: No bootstrap required, opened a new log
I20260807 15:14:30.192646 21585 ts_tablet_manager.cc:1397] T f6dc8f3e065e427086299e5878b671b5 P d76b0cdebcc44295a6d2a7049ef4b030: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 15:14:30.192894 21586 tablet_bootstrap.cc:654] T f6dc8f3e065e427086299e5878b671b5 P 246195cca7b043878aaaf04013697524: Neither blocks nor log segments found. Creating new log.
I20260807 15:14:30.193811 21586 log.cc:824] T f6dc8f3e065e427086299e5878b671b5 P 246195cca7b043878aaaf04013697524: Log is configured to *not* fsync() on all Append() calls
I20260807 15:14:30.194146 21588 tablet_bootstrap.cc:492] T f6dc8f3e065e427086299e5878b671b5 P 732c0e3bda8644778e6306e1aa2eb6d0: Bootstrap starting.
I20260807 15:14:30.194679 21585 raft_consensus.cc:348] T f6dc8f3e065e427086299e5878b671b5 P d76b0cdebcc44295a6d2a7049ef4b030 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "246195cca7b043878aaaf04013697524" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 41283 } } peers { permanent_uuid: "d76b0cdebcc44295a6d2a7049ef4b030" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 34775 } } peers { permanent_uuid: "732c0e3bda8644778e6306e1aa2eb6d0" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 37205 } }
I20260807 15:14:30.194824 21585 raft_consensus.cc:374] T f6dc8f3e065e427086299e5878b671b5 P d76b0cdebcc44295a6d2a7049ef4b030 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:14:30.194856 21585 raft_consensus.cc:729] T f6dc8f3e065e427086299e5878b671b5 P d76b0cdebcc44295a6d2a7049ef4b030 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d76b0cdebcc44295a6d2a7049ef4b030, State: Initialized, Role: FOLLOWER
I20260807 15:14:30.194885 21588 tablet_bootstrap.cc:654] T f6dc8f3e065e427086299e5878b671b5 P 732c0e3bda8644778e6306e1aa2eb6d0: Neither blocks nor log segments found. Creating new log.
I20260807 15:14:30.194898 21586 tablet_bootstrap.cc:492] T f6dc8f3e065e427086299e5878b671b5 P 246195cca7b043878aaaf04013697524: No bootstrap required, opened a new log
I20260807 15:14:30.194957 21586 ts_tablet_manager.cc:1397] T f6dc8f3e065e427086299e5878b671b5 P 246195cca7b043878aaaf04013697524: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260807 15:14:30.195003 21585 consensus_queue.cc:260] T f6dc8f3e065e427086299e5878b671b5 P d76b0cdebcc44295a6d2a7049ef4b030 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "246195cca7b043878aaaf04013697524" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 41283 } } peers { permanent_uuid: "d76b0cdebcc44295a6d2a7049ef4b030" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 34775 } } peers { permanent_uuid: "732c0e3bda8644778e6306e1aa2eb6d0" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 37205 } }
I20260807 15:14:30.195324 21563 heartbeater.cc:503] Master 127.29.194.62:42465 was elected leader, sending a full tablet report...
I20260807 15:14:30.195322 21585 ts_tablet_manager.cc:1428] T f6dc8f3e065e427086299e5878b671b5 P d76b0cdebcc44295a6d2a7049ef4b030: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260807 15:14:30.195468 21588 log.cc:824] T f6dc8f3e065e427086299e5878b671b5 P 732c0e3bda8644778e6306e1aa2eb6d0: Log is configured to *not* fsync() on all Append() calls
I20260807 15:14:30.196615 21588 tablet_bootstrap.cc:492] T f6dc8f3e065e427086299e5878b671b5 P 732c0e3bda8644778e6306e1aa2eb6d0: No bootstrap required, opened a new log
I20260807 15:14:30.196695 21588 ts_tablet_manager.cc:1397] T f6dc8f3e065e427086299e5878b671b5 P 732c0e3bda8644778e6306e1aa2eb6d0: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260807 15:14:30.197474 21586 raft_consensus.cc:348] T f6dc8f3e065e427086299e5878b671b5 P 246195cca7b043878aaaf04013697524 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "246195cca7b043878aaaf04013697524" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 41283 } } peers { permanent_uuid: "d76b0cdebcc44295a6d2a7049ef4b030" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 34775 } } peers { permanent_uuid: "732c0e3bda8644778e6306e1aa2eb6d0" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 37205 } }
I20260807 15:14:30.197693 21586 raft_consensus.cc:374] T f6dc8f3e065e427086299e5878b671b5 P 246195cca7b043878aaaf04013697524 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:14:30.197739 21586 raft_consensus.cc:729] T f6dc8f3e065e427086299e5878b671b5 P 246195cca7b043878aaaf04013697524 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 246195cca7b043878aaaf04013697524, State: Initialized, Role: FOLLOWER
I20260807 15:14:30.197885 21586 consensus_queue.cc:260] T f6dc8f3e065e427086299e5878b671b5 P 246195cca7b043878aaaf04013697524 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "246195cca7b043878aaaf04013697524" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 41283 } } peers { permanent_uuid: "d76b0cdebcc44295a6d2a7049ef4b030" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 34775 } } peers { permanent_uuid: "732c0e3bda8644778e6306e1aa2eb6d0" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 37205 } }
I20260807 15:14:30.198192 21586 ts_tablet_manager.cc:1428] T f6dc8f3e065e427086299e5878b671b5 P 246195cca7b043878aaaf04013697524: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260807 15:14:30.198396 21177 heartbeater.cc:503] Master 127.29.194.62:42465 was elected leader, sending a full tablet report...
I20260807 15:14:30.198818 21588 raft_consensus.cc:348] T f6dc8f3e065e427086299e5878b671b5 P 732c0e3bda8644778e6306e1aa2eb6d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "246195cca7b043878aaaf04013697524" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 41283 } } peers { permanent_uuid: "d76b0cdebcc44295a6d2a7049ef4b030" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 34775 } } peers { permanent_uuid: "732c0e3bda8644778e6306e1aa2eb6d0" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 37205 } }
I20260807 15:14:30.198997 21588 raft_consensus.cc:374] T f6dc8f3e065e427086299e5878b671b5 P 732c0e3bda8644778e6306e1aa2eb6d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:14:30.199051 21588 raft_consensus.cc:729] T f6dc8f3e065e427086299e5878b671b5 P 732c0e3bda8644778e6306e1aa2eb6d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 732c0e3bda8644778e6306e1aa2eb6d0, State: Initialized, Role: FOLLOWER
I20260807 15:14:30.199175 21588 consensus_queue.cc:260] T f6dc8f3e065e427086299e5878b671b5 P 732c0e3bda8644778e6306e1aa2eb6d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "246195cca7b043878aaaf04013697524" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 41283 } } peers { permanent_uuid: "d76b0cdebcc44295a6d2a7049ef4b030" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 34775 } } peers { permanent_uuid: "732c0e3bda8644778e6306e1aa2eb6d0" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 37205 } }
I20260807 15:14:30.199492 21588 ts_tablet_manager.cc:1428] T f6dc8f3e065e427086299e5878b671b5 P 732c0e3bda8644778e6306e1aa2eb6d0: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260807 15:14:30.199501 21381 heartbeater.cc:503] Master 127.29.194.62:42465 was elected leader, sending a full tablet report...
W20260807 15:14:30.234601 21383 tablet.cc:2369] T f6dc8f3e065e427086299e5878b671b5 P 732c0e3bda8644778e6306e1aa2eb6d0: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 15:14:30.327833 21179 tablet.cc:2369] T f6dc8f3e065e427086299e5878b671b5 P 246195cca7b043878aaaf04013697524: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 15:14:30.343076 21593 raft_consensus.cc:482] T f6dc8f3e065e427086299e5878b671b5 P 732c0e3bda8644778e6306e1aa2eb6d0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 15:14:30.343215 21593 raft_consensus.cc:504] T f6dc8f3e065e427086299e5878b671b5 P 732c0e3bda8644778e6306e1aa2eb6d0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "246195cca7b043878aaaf04013697524" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 41283 } } peers { permanent_uuid: "d76b0cdebcc44295a6d2a7049ef4b030" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 34775 } } peers { permanent_uuid: "732c0e3bda8644778e6306e1aa2eb6d0" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 37205 } }
I20260807 15:14:30.343573 21593 leader_election.cc:288] T f6dc8f3e065e427086299e5878b671b5 P 732c0e3bda8644778e6306e1aa2eb6d0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 246195cca7b043878aaaf04013697524 (127.29.194.1:41283), d76b0cdebcc44295a6d2a7049ef4b030 (127.29.194.3:34775)
I20260807 15:14:30.346660 21583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:30.343991 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.194.2:60625 (local address 127.29.194.3:34775)
0807 15:14:30.344065 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0807 15:14:30.344067 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:14:30.344157 (+ 90us) server_negotiation.cc:407] Connection header received
0807 15:14:30.344203 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:14:30.344204 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:14:30.344214 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:14:30.344268 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:14:30.344415 (+ 147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:30.344902 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:30.345894 (+ 992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:30.346277 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:30.346337 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:30.346346 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0807 15:14:30.346538 (+ 192us) server_negotiation.cc:299] Negotiation successful
0807 15:14:30.346580 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260807 15:14:30.346660 21580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:30.343895 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.194.2:37043 (local address 127.29.194.1:41283)
0807 15:14:30.343953 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0807 15:14:30.343955 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:14:30.343966 (+ 11us) server_negotiation.cc:407] Connection header received
0807 15:14:30.343998 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:14:30.344000 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:14:30.344010 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:14:30.344089 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:14:30.344244 (+ 155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:30.344787 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:30.345891 (+ 1104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:30.346275 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:30.346418 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:30.346430 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0807 15:14:30.346549 (+ 119us) server_negotiation.cc:299] Negotiation successful
0807 15:14:30.346593 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260807 15:14:30.346668 21387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:30.343780 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.1:41283 (local address 127.29.194.2:37043)
0807 15:14:30.343852 (+ 72us) negotiation.cc:107] Waiting for socket to connect
0807 15:14:30.343865 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 15:14:30.343916 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:14:30.344095 (+ 179us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:14:30.344096 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:14:30.344098 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:14:30.344171 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:14:30.344173 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:14:30.344867 (+ 694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:14:30.344868 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:14:30.345778 (+ 910us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:14:30.345781 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:14:30.346418 (+ 637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:14:30.346419 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:14:30.346482 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:30.346485 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 15:14:30.346536 (+ 51us) client_negotiation.cc:240] Negotiation successful
0807 15:14:30.346579 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34,"mutex_wait_us":38}
I20260807 15:14:30.346681 21594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:30.343911 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.3:34775 (local address 127.29.194.2:60625)
0807 15:14:30.344086 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0807 15:14:30.344097 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 15:14:30.344154 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:14:30.344280 (+ 126us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:14:30.344281 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:14:30.344282 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:14:30.344346 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:14:30.344348 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:14:30.345019 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:14:30.345020 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:14:30.345802 (+ 782us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:14:30.345804 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:14:30.346378 (+ 574us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:14:30.346379 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:14:30.346461 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:30.346466 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 15:14:30.346513 (+ 47us) client_negotiation.cc:240] Negotiation successful
0807 15:14:30.346585 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":104,"threads_started":1}
I20260807 15:14:30.347060 21499 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f6dc8f3e065e427086299e5878b671b5" candidate_uuid: "732c0e3bda8644778e6306e1aa2eb6d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d76b0cdebcc44295a6d2a7049ef4b030" is_pre_election: true
I20260807 15:14:30.347061 21113 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f6dc8f3e065e427086299e5878b671b5" candidate_uuid: "732c0e3bda8644778e6306e1aa2eb6d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "246195cca7b043878aaaf04013697524" is_pre_election: true
I20260807 15:14:30.347240 21499 raft_consensus.cc:2454] T f6dc8f3e065e427086299e5878b671b5 P d76b0cdebcc44295a6d2a7049ef4b030 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 732c0e3bda8644778e6306e1aa2eb6d0 in term 0.
I20260807 15:14:30.347240 21113 raft_consensus.cc:2454] T f6dc8f3e065e427086299e5878b671b5 P 246195cca7b043878aaaf04013697524 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 732c0e3bda8644778e6306e1aa2eb6d0 in term 0.
I20260807 15:14:30.347496 21212 leader_election.cc:302] T f6dc8f3e065e427086299e5878b671b5 P 732c0e3bda8644778e6306e1aa2eb6d0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 732c0e3bda8644778e6306e1aa2eb6d0, d76b0cdebcc44295a6d2a7049ef4b030; no voters:
I20260807 15:14:30.347630 21593 raft_consensus.cc:2781] T f6dc8f3e065e427086299e5878b671b5 P 732c0e3bda8644778e6306e1aa2eb6d0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 15:14:30.347700 21593 raft_consensus.cc:482] T f6dc8f3e065e427086299e5878b671b5 P 732c0e3bda8644778e6306e1aa2eb6d0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 15:14:30.347723 21593 raft_consensus.cc:3037] T f6dc8f3e065e427086299e5878b671b5 P 732c0e3bda8644778e6306e1aa2eb6d0 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:14:30.348459 21593 raft_consensus.cc:504] T f6dc8f3e065e427086299e5878b671b5 P 732c0e3bda8644778e6306e1aa2eb6d0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "246195cca7b043878aaaf04013697524" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 41283 } } peers { permanent_uuid: "d76b0cdebcc44295a6d2a7049ef4b030" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 34775 } } peers { permanent_uuid: "732c0e3bda8644778e6306e1aa2eb6d0" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 37205 } }
I20260807 15:14:30.348596 21593 leader_election.cc:288] T f6dc8f3e065e427086299e5878b671b5 P 732c0e3bda8644778e6306e1aa2eb6d0 [CANDIDATE]: Term 1 election: Requested vote from peers 246195cca7b043878aaaf04013697524 (127.29.194.1:41283), d76b0cdebcc44295a6d2a7049ef4b030 (127.29.194.3:34775)
I20260807 15:14:30.348811 21499 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f6dc8f3e065e427086299e5878b671b5" candidate_uuid: "732c0e3bda8644778e6306e1aa2eb6d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d76b0cdebcc44295a6d2a7049ef4b030"
I20260807 15:14:30.348812 21113 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f6dc8f3e065e427086299e5878b671b5" candidate_uuid: "732c0e3bda8644778e6306e1aa2eb6d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "246195cca7b043878aaaf04013697524"
I20260807 15:14:30.348888 21113 raft_consensus.cc:3037] T f6dc8f3e065e427086299e5878b671b5 P 246195cca7b043878aaaf04013697524 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:14:30.348892 21499 raft_consensus.cc:3037] T f6dc8f3e065e427086299e5878b671b5 P d76b0cdebcc44295a6d2a7049ef4b030 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:14:30.349694 21499 raft_consensus.cc:2454] T f6dc8f3e065e427086299e5878b671b5 P d76b0cdebcc44295a6d2a7049ef4b030 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 732c0e3bda8644778e6306e1aa2eb6d0 in term 1.
I20260807 15:14:30.349951 21212 leader_election.cc:302] T f6dc8f3e065e427086299e5878b671b5 P 732c0e3bda8644778e6306e1aa2eb6d0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 732c0e3bda8644778e6306e1aa2eb6d0, d76b0cdebcc44295a6d2a7049ef4b030; no voters:
I20260807 15:14:30.350090 21593 raft_consensus.cc:2781] T f6dc8f3e065e427086299e5878b671b5 P 732c0e3bda8644778e6306e1aa2eb6d0 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:14:30.350236 21113 raft_consensus.cc:2454] T f6dc8f3e065e427086299e5878b671b5 P 246195cca7b043878aaaf04013697524 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 732c0e3bda8644778e6306e1aa2eb6d0 in term 1.
I20260807 15:14:30.350389 21593 raft_consensus.cc:686] T f6dc8f3e065e427086299e5878b671b5 P 732c0e3bda8644778e6306e1aa2eb6d0 [term 1 LEADER]: Becoming Leader. State: Replica: 732c0e3bda8644778e6306e1aa2eb6d0, State: Running, Role: LEADER
I20260807 15:14:30.350525 21593 consensus_queue.cc:237] T f6dc8f3e065e427086299e5878b671b5 P 732c0e3bda8644778e6306e1aa2eb6d0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "246195cca7b043878aaaf04013697524" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 41283 } } peers { permanent_uuid: "d76b0cdebcc44295a6d2a7049ef4b030" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 34775 } } peers { permanent_uuid: "732c0e3bda8644778e6306e1aa2eb6d0" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 37205 } }
I20260807 15:14:30.351714 20909 catalog_manager.cc:5495] T f6dc8f3e065e427086299e5878b671b5 P 732c0e3bda8644778e6306e1aa2eb6d0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 732c0e3bda8644778e6306e1aa2eb6d0 (127.29.194.2). New cstate: current_term: 1 leader_uuid: "732c0e3bda8644778e6306e1aa2eb6d0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "246195cca7b043878aaaf04013697524" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 41283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d76b0cdebcc44295a6d2a7049ef4b030" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 34775 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "732c0e3bda8644778e6306e1aa2eb6d0" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 37205 } health_report { overall_health: HEALTHY } } }
I20260807 15:14:30.370487 21584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:30.367438 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57190 (local address 127.29.194.2:37205)
0807 15:14:30.367495 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0807 15:14:30.367497 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:14:30.367504 (+ 7us) server_negotiation.cc:407] Connection header received
0807 15:14:30.367546 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:14:30.367546 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:14:30.367567 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:14:30.367636 (+ 69us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 15:14:30.367767 (+ 131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:30.368290 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:30.369247 (+ 957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:30.369651 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:30.369812 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:30.370056 (+ 244us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 15:14:30.370207 (+ 151us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 15:14:30.370296 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0807 15:14:30.370318 (+ 22us) server_negotiation.cc:299] Negotiation successful
0807 15:14:30.370349 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260807 15:14:30.372638 21499 raft_consensus.cc:1261] T f6dc8f3e065e427086299e5878b671b5 P d76b0cdebcc44295a6d2a7049ef4b030 [term 1 FOLLOWER]: Refusing update from remote peer 732c0e3bda8644778e6306e1aa2eb6d0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:14:30.372779 21113 raft_consensus.cc:1261] T f6dc8f3e065e427086299e5878b671b5 P 246195cca7b043878aaaf04013697524 [term 1 FOLLOWER]: Refusing update from remote peer 732c0e3bda8644778e6306e1aa2eb6d0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:14:30.373059 21595 consensus_queue.cc:1035] T f6dc8f3e065e427086299e5878b671b5 P 732c0e3bda8644778e6306e1aa2eb6d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d76b0cdebcc44295a6d2a7049ef4b030" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 34775 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:14:30.373237 21593 consensus_queue.cc:1035] T f6dc8f3e065e427086299e5878b671b5 P 732c0e3bda8644778e6306e1aa2eb6d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "246195cca7b043878aaaf04013697524" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 41283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:14:30.378549 21600 mvcc.cc:205] Tried to move back new op lower bound from 7315929785843486720 to 7315929785757224960. Current Snapshot: MvccSnapshot[applied={T|T < 7315929785843486720 or (T in {7315929785843486720})}]
I20260807 15:14:30.379925 20908 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41860:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 15:14:30.380060 20908 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41860:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 15:14:30.380904 20908 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4e95b6a1e91e4fc0b95dbb205a0d0d4a: Sending DeleteTablet for 3 replicas of tablet f6dc8f3e065e427086299e5878b671b5
I20260807 15:14:30.381246 30472 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7fZSIL/build/release/bin/kudu with pid 20994
I20260807 15:14:30.381256 21083 tablet_service.cc:1514] Processing DeleteTablet for tablet f6dc8f3e065e427086299e5878b671b5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:14:30 UTC) from {username='kudu', principal='kudu/127.29.194.62@KRBTEST.COM'} at 127.0.0.1:52316
I20260807 15:14:30.381278 21276 tablet_service.cc:1514] Processing DeleteTablet for tablet f6dc8f3e065e427086299e5878b671b5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:14:30 UTC) from {username='kudu', principal='kudu/127.29.194.62@KRBTEST.COM'} at 127.0.0.1:57186
I20260807 15:14:30.381268 21473 tablet_service.cc:1514] Processing DeleteTablet for tablet f6dc8f3e065e427086299e5878b671b5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:14:30 UTC) from {username='kudu', principal='kudu/127.29.194.62@KRBTEST.COM'} at 127.0.0.1:44646
I20260807 15:14:30.385279 21611 tablet_replica.cc:330] T f6dc8f3e065e427086299e5878b671b5 P d76b0cdebcc44295a6d2a7049ef4b030: stopping tablet replica
I20260807 15:14:30.385433 21611 raft_consensus.cc:2229] T f6dc8f3e065e427086299e5878b671b5 P d76b0cdebcc44295a6d2a7049ef4b030 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 15:14:30.385617 21611 raft_consensus.cc:2258] T f6dc8f3e065e427086299e5878b671b5 P d76b0cdebcc44295a6d2a7049ef4b030 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:14:30.386183 21611 ts_tablet_manager.cc:1905] T f6dc8f3e065e427086299e5878b671b5 P d76b0cdebcc44295a6d2a7049ef4b030: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:14:30.388306 21611 ts_tablet_manager.cc:1918] T f6dc8f3e065e427086299e5878b671b5 P d76b0cdebcc44295a6d2a7049ef4b030: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 15:14:30.388384 21611 log.cc:1196] T f6dc8f3e065e427086299e5878b671b5 P d76b0cdebcc44295a6d2a7049ef4b030: Deleting WAL directory at /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115480851507-30472-0/minicluster-data/ts-2/wal/wals/f6dc8f3e065e427086299e5878b671b5
I20260807 15:14:30.388661 21611 ts_tablet_manager.cc:1939] T f6dc8f3e065e427086299e5878b671b5 P d76b0cdebcc44295a6d2a7049ef4b030: Deleting consensus metadata
I20260807 15:14:30.389140 20882 catalog_manager.cc:4841] TS d76b0cdebcc44295a6d2a7049ef4b030 (127.29.194.3:34775): tablet f6dc8f3e065e427086299e5878b671b5 (table test-table [id=115c5af53e0c426fbac39efc831e81f8]) successfully deleted
W20260807 15:14:30.391808 20882 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.29.194.1:41283 (error 108)
I20260807 15:14:30.392275 30472 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7fZSIL/build/release/bin/kudu with pid 21184
I20260807 15:14:30.392426 21582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:30.392273 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.1:41283 (local address 127.0.0.1:52326)
0807 15:14:30.392333 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0807 15:14:30.392379 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.194.1:41283: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260807 15:14:30.403005 20882 catalog_manager.cc:4584] TS 246195cca7b043878aaaf04013697524 (127.29.194.1:41283): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet f6dc8f3e065e427086299e5878b671b5: Network error: Client connection negotiation failed: client connection to 127.29.194.1:41283: connect: Connection refused (error 111)
I20260807 15:14:30.407568 21582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:30.407436 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.2:37205 (local address 127.0.0.1:57206)
0807 15:14:30.407493 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0807 15:14:30.407522 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.194.2:37205: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
I20260807 15:14:30.407840 30472 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7fZSIL/build/release/bin/kudu with pid 21390
I20260807 15:14:30.420962 30472 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7fZSIL/build/release/bin/kudu with pid 20860
I20260807 15:14:50.431156 30472 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7fZSIL/build/release/bin/kudu
/tmp/dist-test-task7fZSIL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115480851507-30472-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115480851507-30472-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115480851507-30472-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115480851507-30472-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.194.62:42465
--webserver_interface=127.29.194.62
--webserver_port=41213
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.62.keytab
--principal=kudu/127.29.194.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.194.20:40443
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.194.62:42465
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115480851507-30472-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115480851507-30472-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115480851507-30472-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:14:50.522131 21614 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:14:50.522387 21614 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:14:50.522414 21614 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:14:50.523622 21614 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260807 15:14:50.523669 21614 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260807 15:14:50.523681 21614 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260807 15:14:50.523692 21614 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:14:50.523703 21614 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:14:50.523715 21614 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:14:50.523725 21614 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:14:50.523736 21614 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:14:50.525564 21614 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.194.20:40443
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115480851507-30472-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115480851507-30472-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.29.194.62:42465
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.62.keytab
--principal=kudu/127.29.194.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.194.62:42465
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115480851507-30472-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.194.62
--webserver_port=41213
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21614
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115480851507-30472-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:14:50.525883 21614 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:14:50.526151 21614 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:14:50.537819 21622 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:14:50.537930 21626 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:14:50.538187 21614 server_base.cc:1034] running on GCE node
W20260807 15:14:50.537935 21623 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:14:50.538558 21614 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:14:50.538879 21614 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:14:50.540037 21614 hybrid_clock.cc:648] HybridClock initialized: now 1786115690540028 us; error 33 us; skew 500 ppm
Aug 07 15:14:50 dist-test-slave-42z9 krb5kdc[20839](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115690, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.194.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:14:50.548936 21614 init.cc:377] Logged in from keytab as kudu/127.29.194.62@KRBTEST.COM (short username kudu)
I20260807 15:14:50.549608 21614 webserver.cc:460] Webserver started at http://127.29.194.62:41213/ using document root <none> and password file <none>
I20260807 15:14:50.549865 21614 fs_manager.cc:359] Metadata directory not provided
I20260807 15:14:50.549927 21614 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:14:50.551676 21614 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260807 15:14:50.552570 21635 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:14:50.552773 21614 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 15:14:50.552886 21614 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115480851507-30472-0/minicluster-data/master-0/data,/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115480851507-30472-0/minicluster-data/master-0/wal
uuid: "4e95b6a1e91e4fc0b95dbb205a0d0d4a"
format_stamp: "Formatted at 2026-08-07 15:14:29 on dist-test-slave-42z9"
server_key: "964c398495f7750df66021b9acd44de6"
server_key_iv: "79b9c8fbbc418736323465fe6474d8fc"
server_key_version: "encryptionkey@0"
I20260807 15:14:50.553344 21614 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115480851507-30472-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115480851507-30472-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115480851507-30472-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:14:50.565009 21614 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:14:50.568174 21614 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:14:50.568385 21614 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:14:50.574501 21614 rpc_server.cc:307] RPC server started. Bound to: 127.29.194.62:42465
I20260807 15:14:50.574548 21709 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.194.62:42465 every 8 connection(s)
I20260807 15:14:50.575003 21614 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115480851507-30472-0/minicluster-data/master-0/data/info.pb
I20260807 15:14:50.576728 21712 sys_catalog.cc:263] Verifying existing consensus state
I20260807 15:14:50.577507 21712 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4e95b6a1e91e4fc0b95dbb205a0d0d4a: Bootstrap starting.
I20260807 15:14:50.578682 30472 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7fZSIL/build/release/bin/kudu as pid 21614
I20260807 15:14:50.579000 30472 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7fZSIL/build/release/bin/kudu
/tmp/dist-test-task7fZSIL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115480851507-30472-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115480851507-30472-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115480851507-30472-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115480851507-30472-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.194.1:41283
--local_ip_for_outbound_sockets=127.29.194.1
--tserver_master_addrs=127.29.194.62:42465
--webserver_port=40433
--webserver_interface=127.29.194.1
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.1.keytab
--principal=kudu/127.29.194.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.194.20:40443
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115480851507-30472-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115480851507-30472-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115480851507-30472-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 15:14:50.587857 21712 log.cc:824] T 00000000000000000000000000000000 P 4e95b6a1e91e4fc0b95dbb205a0d0d4a: Log is configured to *not* fsync() on all Append() calls
I20260807 15:14:50.590406 21712 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4e95b6a1e91e4fc0b95dbb205a0d0d4a: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 15:14:50.590763 21712 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4e95b6a1e91e4fc0b95dbb205a0d0d4a: Bootstrap complete.
I20260807 15:14:50.593108 21712 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4e95b6a1e91e4fc0b95dbb205a0d0d4a [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e95b6a1e91e4fc0b95dbb205a0d0d4a" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 42465 } }
I20260807 15:14:50.593433 21712 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4e95b6a1e91e4fc0b95dbb205a0d0d4a [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e95b6a1e91e4fc0b95dbb205a0d0d4a, State: Initialized, Role: FOLLOWER
I20260807 15:14:50.593657 21712 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4e95b6a1e91e4fc0b95dbb205a0d0d4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e95b6a1e91e4fc0b95dbb205a0d0d4a" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 42465 } }
I20260807 15:14:50.593737 21712 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4e95b6a1e91e4fc0b95dbb205a0d0d4a [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:14:50.593791 21712 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4e95b6a1e91e4fc0b95dbb205a0d0d4a [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:14:50.593838 21712 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4e95b6a1e91e4fc0b95dbb205a0d0d4a [term 1 FOLLOWER]: Advancing to term 2
I20260807 15:14:50.595001 21712 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4e95b6a1e91e4fc0b95dbb205a0d0d4a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e95b6a1e91e4fc0b95dbb205a0d0d4a" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 42465 } }
I20260807 15:14:50.595110 21712 leader_election.cc:302] T 00000000000000000000000000000000 P 4e95b6a1e91e4fc0b95dbb205a0d0d4a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4e95b6a1e91e4fc0b95dbb205a0d0d4a; no voters:
I20260807 15:14:50.595288 21712 leader_election.cc:288] T 00000000000000000000000000000000 P 4e95b6a1e91e4fc0b95dbb205a0d0d4a [CANDIDATE]: Term 2 election: Requested vote from peers
I20260807 15:14:50.595345 21720 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4e95b6a1e91e4fc0b95dbb205a0d0d4a [term 2 FOLLOWER]: Leader election won for term 2
I20260807 15:14:50.595472 21720 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4e95b6a1e91e4fc0b95dbb205a0d0d4a [term 2 LEADER]: Becoming Leader. State: Replica: 4e95b6a1e91e4fc0b95dbb205a0d0d4a, State: Running, Role: LEADER
I20260807 15:14:50.595590 21712 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4e95b6a1e91e4fc0b95dbb205a0d0d4a [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:14:50.595649 21720 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4e95b6a1e91e4fc0b95dbb205a0d0d4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e95b6a1e91e4fc0b95dbb205a0d0d4a" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 42465 } }
I20260807 15:14:50.595976 21722 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4e95b6a1e91e4fc0b95dbb205a0d0d4a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "4e95b6a1e91e4fc0b95dbb205a0d0d4a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e95b6a1e91e4fc0b95dbb205a0d0d4a" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 42465 } } }
I20260807 15:14:50.596200 21722 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4e95b6a1e91e4fc0b95dbb205a0d0d4a [sys.catalog]: This master's current role is: LEADER
I20260807 15:14:50.596141 21720 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4e95b6a1e91e4fc0b95dbb205a0d0d4a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4e95b6a1e91e4fc0b95dbb205a0d0d4a. Latest consensus state: current_term: 2 leader_uuid: "4e95b6a1e91e4fc0b95dbb205a0d0d4a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e95b6a1e91e4fc0b95dbb205a0d0d4a" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 42465 } } }
I20260807 15:14:50.596422 21720 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4e95b6a1e91e4fc0b95dbb205a0d0d4a [sys.catalog]: This master's current role is: LEADER
I20260807 15:14:50.597431 21749 catalog_manager.cc:1259] Loaded cluster ID: e054a19df1ae4e99a1ba8f154ecc4884
I20260807 15:14:50.597509 21749 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 4e95b6a1e91e4fc0b95dbb205a0d0d4a: loading cluster ID for follower catalog manager: success
I20260807 15:14:50.598338 21749 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 4e95b6a1e91e4fc0b95dbb205a0d0d4a: acquiring CA information for follower catalog manager: success
I20260807 15:14:50.598711 21749 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 4e95b6a1e91e4fc0b95dbb205a0d0d4a: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 15:14:50.598771 21737 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:14:50.599097 21737 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:14:50.599169 21737 catalog_manager.cc:1259] Loaded cluster ID: e054a19df1ae4e99a1ba8f154ecc4884
I20260807 15:14:50.599195 21737 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:14:50.599328 21737 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:14:50.599421 21737 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 4e95b6a1e91e4fc0b95dbb205a0d0d4a: Loaded TSK: 0
I20260807 15:14:50.608366 21737 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4e95b6a1e91e4fc0b95dbb205a0d0d4a: Generated new TSK 1
I20260807 15:14:50.608645 21737 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260807 15:14:50.685148 21715 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:14:50.685402 21715 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:14:50.685431 21715 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:14:50.686777 21715 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:14:50.686839 21715 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:14:50.686869 21715 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.194.1
I20260807 15:14:50.688534 21715 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.194.20:40443
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115480851507-30472-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115480851507-30472-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.1.keytab
--principal=kudu/127.29.194.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.194.1:41283
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115480851507-30472-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.194.1
--webserver_port=40433
--tserver_master_addrs=127.29.194.62:42465
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21715
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.194.1
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115480851507-30472-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:14:50.688809 21715 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:14:50.689059 21715 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:14:50.694463 21765 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:14:50.694489 21761 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:14:50.694495 21762 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:14:50.695102 21715 server_base.cc:1034] running on GCE node
I20260807 15:14:50.695318 21715 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:14:50.695574 21715 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:14:50.696806 21715 hybrid_clock.cc:648] HybridClock initialized: now 1786115690696775 us; error 48 us; skew 500 ppm
Aug 07 15:14:50 dist-test-slave-42z9 krb5kdc[20839](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115690, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.194.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:14:50.699448 21715 init.cc:377] Logged in from keytab as kudu/127.29.194.1@KRBTEST.COM (short username kudu)
I20260807 15:14:50.699947 21715 webserver.cc:460] Webserver started at http://127.29.194.1:40433/ using document root <none> and password file <none>
I20260807 15:14:50.700176 21715 fs_manager.cc:359] Metadata directory not provided
I20260807 15:14:50.700244 21715 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:14:50.701776 21715 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:14:50.702493 21777 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:14:50.702656 21715 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:14:50.702747 21715 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115480851507-30472-0/minicluster-data/ts-0/data,/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115480851507-30472-0/minicluster-data/ts-0/wal
uuid: "246195cca7b043878aaaf04013697524"
format_stamp: "Formatted at 2026-08-07 15:14:29 on dist-test-slave-42z9"
server_key: "11e8656c00110a93fcf480a485a0826c"
server_key_iv: "4716fd9bfb72a0e363b1a5b3a1dc54f1"
server_key_version: "encryptionkey@0"
I20260807 15:14:50.703125 21715 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115480851507-30472-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115480851507-30472-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115480851507-30472-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:14:50.723030 21715 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:14:50.725634 21715 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:14:50.725801 21715 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:14:50.726090 21715 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:14:50.726686 21785 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260807 15:14:50.728946 21715 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260807 15:14:50.729046 21715 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.003s user 0.000s sys 0.000s
I20260807 15:14:50.729091 21715 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260807 15:14:50.729910 21715 ts_tablet_manager.cc:610] Registered 1 tablets
I20260807 15:14:50.729954 21715 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.001s sys 0.000s
I20260807 15:14:50.730089 21785 tablet_bootstrap.cc:492] T f6dc8f3e065e427086299e5878b671b5 P 246195cca7b043878aaaf04013697524: Bootstrap starting.
I20260807 15:14:50.738745 21715 rpc_server.cc:307] RPC server started. Bound to: 127.29.194.1:41283
I20260807 15:14:50.739254 21715 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115480851507-30472-0/minicluster-data/ts-0/data/info.pb
I20260807 15:14:50.742331 21945 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.194.1:41283 every 8 connection(s)
I20260807 15:14:50.744218 21785 log.cc:824] T f6dc8f3e065e427086299e5878b671b5 P 246195cca7b043878aaaf04013697524: Log is configured to *not* fsync() on all Append() calls
I20260807 15:14:50.747622 30472 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7fZSIL/build/release/bin/kudu as pid 21715
I20260807 15:14:50.747931 21785 tablet_bootstrap.cc:492] T f6dc8f3e065e427086299e5878b671b5 P 246195cca7b043878aaaf04013697524: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 15:14:50.747956 30472 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7fZSIL/build/release/bin/kudu
/tmp/dist-test-task7fZSIL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115480851507-30472-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115480851507-30472-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115480851507-30472-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115480851507-30472-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.194.2:37205
--local_ip_for_outbound_sockets=127.29.194.2
--tserver_master_addrs=127.29.194.62:42465
--webserver_port=36107
--webserver_interface=127.29.194.2
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.2.keytab
--principal=kudu/127.29.194.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.194.20:40443
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115480851507-30472-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115480851507-30472-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115480851507-30472-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 15:14:50.748212 21785 tablet_bootstrap.cc:492] T f6dc8f3e065e427086299e5878b671b5 P 246195cca7b043878aaaf04013697524: Bootstrap complete.
I20260807 15:14:50.749272 21785 ts_tablet_manager.cc:1397] T f6dc8f3e065e427086299e5878b671b5 P 246195cca7b043878aaaf04013697524: Time spent bootstrapping tablet: real 0.019s user 0.000s sys 0.013s
I20260807 15:14:50.750771 21785 raft_consensus.cc:348] T f6dc8f3e065e427086299e5878b671b5 P 246195cca7b043878aaaf04013697524 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "246195cca7b043878aaaf04013697524" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 41283 } } peers { permanent_uuid: "d76b0cdebcc44295a6d2a7049ef4b030" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 34775 } } peers { permanent_uuid: "732c0e3bda8644778e6306e1aa2eb6d0" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 37205 } }
I20260807 15:14:50.751046 21785 raft_consensus.cc:729] T f6dc8f3e065e427086299e5878b671b5 P 246195cca7b043878aaaf04013697524 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 246195cca7b043878aaaf04013697524, State: Initialized, Role: FOLLOWER
I20260807 15:14:50.751186 21785 consensus_queue.cc:260] T f6dc8f3e065e427086299e5878b671b5 P 246195cca7b043878aaaf04013697524 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "246195cca7b043878aaaf04013697524" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 41283 } } peers { permanent_uuid: "d76b0cdebcc44295a6d2a7049ef4b030" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 34775 } } peers { permanent_uuid: "732c0e3bda8644778e6306e1aa2eb6d0" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 37205 } }
I20260807 15:14:50.751490 21785 ts_tablet_manager.cc:1428] T f6dc8f3e065e427086299e5878b671b5 P 246195cca7b043878aaaf04013697524: Time spent starting tablet: real 0.002s user 0.002s sys 0.004s
Aug 07 15:14:50 dist-test-slave-42z9 krb5kdc[20839](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115690, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.194.1@KRBTEST.COM for kudu/127.29.194.62@KRBTEST.COM
I20260807 15:14:50.761559 21949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:50.745974 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:42465 (local address 127.29.194.1:34445)
0807 15:14:50.746620 (+ 646us) negotiation.cc:107] Waiting for socket to connect
0807 15:14:50.746664 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0807 15:14:50.748152 (+ 1488us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:14:50.749067 (+ 915us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:14:50.749091 (+ 24us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:14:50.749668 (+ 577us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:14:50.750087 (+ 419us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:14:50.750095 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:14:50.751885 (+ 1790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:14:50.751886 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:14:50.752711 (+ 825us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:14:50.752714 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:14:50.753552 (+ 838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:14:50.753553 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:14:50.753670 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:50.754236 (+ 566us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:14:50.754245 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:14:50.756296 (+ 2051us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:14:50.759043 (+ 2747us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:14:50.759060 (+ 17us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:14:50.759063 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:14:50.759483 (+ 420us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:14:50.759940 (+ 457us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:14:50.759943 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:14:50.759943 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:14:50.760149 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:14:50.760732 (+ 583us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:14:50.760742 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:14:50.760931 (+ 189us) client_negotiation.cc:769] Sending connection context
0807 15:14:50.761053 (+ 122us) client_negotiation.cc:240] Negotiation successful
0807 15:14:50.761279 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":597,"thread_start_us":90,"threads_started":1}
I20260807 15:14:50.761660 21950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:50.746396 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.194.1:34445 (local address 127.29.194.62:42465)
0807 15:14:50.746844 (+ 448us) server_negotiation.cc:206] Beginning negotiation
0807 15:14:50.746854 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 15:14:50.746903 (+ 49us) server_negotiation.cc:407] Connection header received
0807 15:14:50.748537 (+ 1634us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:14:50.748544 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:14:50.748636 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:14:50.748801 (+ 165us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:14:50.750273 (+ 1472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:50.751740 (+ 1467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:50.752822 (+ 1082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:50.753434 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:50.753569 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:50.756521 (+ 2952us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:14:50.756527 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:14:50.756528 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:14:50.756545 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:14:50.758881 (+ 2336us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:14:50.759612 (+ 731us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:14:50.759616 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:14:50.759616 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:14:50.759777 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:14:50.760234 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:14:50.760235 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:14:50.760235 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:14:50.760613 (+ 378us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:14:50.760718 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0807 15:14:50.761162 (+ 444us) server_negotiation.cc:299] Negotiation successful
0807 15:14:50.761361 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":393,"thread_start_us":92,"threads_started":1}
I20260807 15:14:50.762637 21946 heartbeater.cc:347] Connected to a master server at 127.29.194.62:42465
I20260807 15:14:50.762787 21946 heartbeater.cc:464] Registering TS with master...
I20260807 15:14:50.762993 21946 heartbeater.cc:511] Master 127.29.194.62:42465 requested a full tablet report, sending...
I20260807 15:14:50.763859 21659 ts_manager.cc:194] Registered new tserver with Master: 246195cca7b043878aaaf04013697524 (127.29.194.1:41283)
I20260807 15:14:50.765210 21659 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.194.1@KRBTEST.COM'} at 127.29.194.1:34445
I20260807 15:14:50.766393 21946 heartbeater.cc:503] Master 127.29.194.62:42465 was elected leader, sending a full tablet report...
Aug 07 15:14:50 dist-test-slave-42z9 krb5kdc[20839](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115690, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.194.62@KRBTEST.COM for kudu/127.29.194.1@KRBTEST.COM
I20260807 15:14:50.773896 21955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:50.764746 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.1:41283 (local address 127.0.0.1:40132)
0807 15:14:50.765000 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0807 15:14:50.765041 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0807 15:14:50.765149 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:14:50.765746 (+ 597us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:14:50.765751 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:14:50.766074 (+ 323us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:14:50.766152 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:14:50.766155 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:14:50.767391 (+ 1236us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:14:50.767393 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:14:50.768063 (+ 670us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:14:50.768067 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:14:50.768550 (+ 483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:14:50.768552 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:14:50.768635 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:50.769132 (+ 497us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:14:50.769138 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:14:50.770777 (+ 1639us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:14:50.772698 (+ 1921us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:14:50.772706 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:14:50.772708 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:14:50.772927 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:14:50.773127 (+ 200us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:14:50.773127 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:14:50.773127 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:14:50.773180 (+ 53us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:14:50.773558 (+ 378us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:14:50.773565 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:14:50.773616 (+ 51us) client_negotiation.cc:769] Sending connection context
0807 15:14:50.773688 (+ 72us) client_negotiation.cc:240] Negotiation successful
0807 15:14:50.773802 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":211,"thread_start_us":155,"threads_started":1}
I20260807 15:14:50.773962 21956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:50.765214 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40132 (local address 127.29.194.1:41283)
0807 15:14:50.765500 (+ 286us) server_negotiation.cc:206] Beginning negotiation
0807 15:14:50.765504 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:14:50.765514 (+ 10us) server_negotiation.cc:407] Connection header received
0807 15:14:50.765558 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:14:50.765561 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:14:50.765581 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:14:50.765658 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:14:50.766247 (+ 589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:50.767294 (+ 1047us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:50.768145 (+ 851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:50.768475 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:50.768537 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:50.770870 (+ 2333us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:14:50.770881 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:14:50.770882 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:14:50.770890 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:14:50.772602 (+ 1712us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:14:50.772998 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:14:50.773002 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:14:50.773002 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:14:50.773070 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:14:50.773226 (+ 156us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:14:50.773226 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:14:50.773226 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:14:50.773417 (+ 191us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:14:50.773549 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0807 15:14:50.773738 (+ 189us) server_negotiation.cc:299] Negotiation successful
0807 15:14:50.773855 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":201,"thread_start_us":109,"threads_started":1}
I20260807 15:14:50.774416 21836 tablet_service.cc:1514] Processing DeleteTablet for tablet f6dc8f3e065e427086299e5878b671b5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:14:30 UTC) from {username='kudu', principal='kudu/127.29.194.62@KRBTEST.COM'} at 127.0.0.1:40132
I20260807 15:14:50.774955 21957 tablet_replica.cc:330] stopping tablet replica
I20260807 15:14:50.775050 21957 raft_consensus.cc:2229] T f6dc8f3e065e427086299e5878b671b5 P 246195cca7b043878aaaf04013697524 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 15:14:50.775121 21957 raft_consensus.cc:2258] T f6dc8f3e065e427086299e5878b671b5 P 246195cca7b043878aaaf04013697524 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:14:50.775450 21957 ts_tablet_manager.cc:1905] T f6dc8f3e065e427086299e5878b671b5 P 246195cca7b043878aaaf04013697524: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:14:50.777746 21957 ts_tablet_manager.cc:1918] T f6dc8f3e065e427086299e5878b671b5 P 246195cca7b043878aaaf04013697524: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 15:14:50.777875 21957 log.cc:1196] T f6dc8f3e065e427086299e5878b671b5 P 246195cca7b043878aaaf04013697524: Deleting WAL directory at /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115480851507-30472-0/minicluster-data/ts-0/wal/wals/f6dc8f3e065e427086299e5878b671b5
I20260807 15:14:50.778280 21957 ts_tablet_manager.cc:1939] T f6dc8f3e065e427086299e5878b671b5 P 246195cca7b043878aaaf04013697524: Deleting consensus metadata
I20260807 15:14:50.778888 21638 catalog_manager.cc:4841] TS 246195cca7b043878aaaf04013697524 (127.29.194.1:41283): tablet f6dc8f3e065e427086299e5878b671b5 (table test-table [id=115c5af53e0c426fbac39efc831e81f8]) successfully deleted
W20260807 15:14:50.841305 21951 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:14:50.841565 21951 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:14:50.841602 21951 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:14:50.842813 21951 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:14:50.842861 21951 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:14:50.842900 21951 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.194.2
I20260807 15:14:50.844460 21951 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.194.20:40443
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115480851507-30472-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115480851507-30472-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.2.keytab
--principal=kudu/127.29.194.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.194.2:37205
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115480851507-30472-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.194.2
--webserver_port=36107
--tserver_master_addrs=127.29.194.62:42465
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21951
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.194.2
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115480851507-30472-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:14:50.844722 21951 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:14:50.844954 21951 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:14:50.850461 21965 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:14:50.850605 21951 server_base.cc:1034] running on GCE node
W20260807 15:14:50.850440 21966 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:14:50.850411 21968 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:14:50.850889 21951 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:14:50.851135 21951 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:14:50.852290 21951 hybrid_clock.cc:648] HybridClock initialized: now 1786115690852275 us; error 26 us; skew 500 ppm
Aug 07 15:14:50 dist-test-slave-42z9 krb5kdc[20839](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115690, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.194.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:14:50.854956 21951 init.cc:377] Logged in from keytab as kudu/127.29.194.2@KRBTEST.COM (short username kudu)
I20260807 15:14:50.855427 21951 webserver.cc:460] Webserver started at http://127.29.194.2:36107/ using document root <none> and password file <none>
I20260807 15:14:50.855660 21951 fs_manager.cc:359] Metadata directory not provided
I20260807 15:14:50.855729 21951 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:14:50.857110 21951 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:14:50.857950 21978 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:14:50.858109 21951 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260807 15:14:50.858194 21951 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115480851507-30472-0/minicluster-data/ts-1/data,/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115480851507-30472-0/minicluster-data/ts-1/wal
uuid: "732c0e3bda8644778e6306e1aa2eb6d0"
format_stamp: "Formatted at 2026-08-07 15:14:29 on dist-test-slave-42z9"
server_key: "4127a96f8d5a6dfc46b331de9e58b49f"
server_key_iv: "c45eaa31b7403e481f8cbc847079cf50"
server_key_version: "encryptionkey@0"
I20260807 15:14:50.858554 21951 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115480851507-30472-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115480851507-30472-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115480851507-30472-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:14:50.873006 21951 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:14:50.875701 21951 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:14:50.875864 21951 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:14:50.876111 21951 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:14:50.876613 21987 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260807 15:14:50.878288 21951 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260807 15:14:50.878412 21951 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260807 15:14:50.878463 21951 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260807 15:14:50.879385 21951 ts_tablet_manager.cc:610] Registered 1 tablets
I20260807 15:14:50.879443 21951 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260807 15:14:50.879485 21987 tablet_bootstrap.cc:492] T f6dc8f3e065e427086299e5878b671b5 P 732c0e3bda8644778e6306e1aa2eb6d0: Bootstrap starting.
I20260807 15:14:50.886533 21951 rpc_server.cc:307] RPC server started. Bound to: 127.29.194.2:37205
I20260807 15:14:50.886626 22141 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.194.2:37205 every 8 connection(s)
I20260807 15:14:50.886963 21951 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115480851507-30472-0/minicluster-data/ts-1/data/info.pb
I20260807 15:14:50.887320 30472 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7fZSIL/build/release/bin/kudu as pid 21951
I20260807 15:14:50.887593 30472 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7fZSIL/build/release/bin/kudu
/tmp/dist-test-task7fZSIL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115480851507-30472-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115480851507-30472-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115480851507-30472-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115480851507-30472-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.194.3:34775
--local_ip_for_outbound_sockets=127.29.194.3
--tserver_master_addrs=127.29.194.62:42465
--webserver_port=40145
--webserver_interface=127.29.194.3
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.3.keytab
--principal=kudu/127.29.194.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.194.20:40443
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115480851507-30472-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115480851507-30472-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115480851507-30472-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 15:14:50.888960 21987 log.cc:824] T f6dc8f3e065e427086299e5878b671b5 P 732c0e3bda8644778e6306e1aa2eb6d0: Log is configured to *not* fsync() on all Append() calls
I20260807 15:14:50.891424 21987 tablet_bootstrap.cc:492] T f6dc8f3e065e427086299e5878b671b5 P 732c0e3bda8644778e6306e1aa2eb6d0: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 15:14:50.891737 21987 tablet_bootstrap.cc:492] T f6dc8f3e065e427086299e5878b671b5 P 732c0e3bda8644778e6306e1aa2eb6d0: Bootstrap complete.
I20260807 15:14:50.892849 21987 ts_tablet_manager.cc:1397] T f6dc8f3e065e427086299e5878b671b5 P 732c0e3bda8644778e6306e1aa2eb6d0: Time spent bootstrapping tablet: real 0.013s user 0.007s sys 0.005s
Aug 07 15:14:50 dist-test-slave-42z9 krb5kdc[20839](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115690, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.194.2@KRBTEST.COM for kudu/127.29.194.62@KRBTEST.COM
I20260807 15:14:50.894021 21987 raft_consensus.cc:348] T f6dc8f3e065e427086299e5878b671b5 P 732c0e3bda8644778e6306e1aa2eb6d0 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "246195cca7b043878aaaf04013697524" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 41283 } } peers { permanent_uuid: "d76b0cdebcc44295a6d2a7049ef4b030" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 34775 } } peers { permanent_uuid: "732c0e3bda8644778e6306e1aa2eb6d0" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 37205 } }
I20260807 15:14:50.894353 21987 raft_consensus.cc:729] T f6dc8f3e065e427086299e5878b671b5 P 732c0e3bda8644778e6306e1aa2eb6d0 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 732c0e3bda8644778e6306e1aa2eb6d0, State: Initialized, Role: FOLLOWER
I20260807 15:14:50.894500 21987 consensus_queue.cc:260] T f6dc8f3e065e427086299e5878b671b5 P 732c0e3bda8644778e6306e1aa2eb6d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "246195cca7b043878aaaf04013697524" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 41283 } } peers { permanent_uuid: "d76b0cdebcc44295a6d2a7049ef4b030" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 34775 } } peers { permanent_uuid: "732c0e3bda8644778e6306e1aa2eb6d0" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 37205 } }
I20260807 15:14:50.894780 21987 ts_tablet_manager.cc:1428] T f6dc8f3e065e427086299e5878b671b5 P 732c0e3bda8644778e6306e1aa2eb6d0: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260807 15:14:50.897609 21950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:50.887839 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.194.2:55093 (local address 127.29.194.62:42465)
0807 15:14:50.887888 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0807 15:14:50.887890 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:14:50.888232 (+ 342us) server_negotiation.cc:407] Connection header received
0807 15:14:50.888864 (+ 632us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:14:50.888866 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:14:50.888878 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:14:50.888949 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:14:50.889780 (+ 831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:50.890541 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:50.891754 (+ 1213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:50.892026 (+ 272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:50.892088 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:50.894718 (+ 2630us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:14:50.894719 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:14:50.894720 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:14:50.894728 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:14:50.896122 (+ 1394us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:14:50.896601 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:14:50.896602 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:14:50.896603 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:14:50.896660 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:14:50.896882 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:14:50.896883 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:14:50.896884 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:14:50.897013 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:14:50.897075 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0807 15:14:50.897364 (+ 289us) server_negotiation.cc:299] Negotiation successful
0807 15:14:50.897515 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260807 15:14:50.897730 22144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:50.887795 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:42465 (local address 127.29.194.2:55093)
0807 15:14:50.888012 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0807 15:14:50.888064 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0807 15:14:50.888736 (+ 672us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:14:50.889020 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:14:50.889024 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:14:50.889412 (+ 388us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:14:50.889682 (+ 270us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:14:50.889688 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:14:50.890705 (+ 1017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:14:50.890706 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:14:50.891657 (+ 951us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:14:50.891661 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:14:50.892108 (+ 447us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:14:50.892109 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:14:50.892239 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:50.892746 (+ 507us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:14:50.892755 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:14:50.894587 (+ 1832us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:14:50.896237 (+ 1650us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:14:50.896244 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:14:50.896247 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:14:50.896530 (+ 283us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:14:50.896716 (+ 186us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:14:50.896717 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:14:50.896717 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:14:50.896825 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:14:50.897073 (+ 248us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:14:50.897082 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:14:50.897221 (+ 139us) client_negotiation.cc:769] Sending connection context
0807 15:14:50.897343 (+ 122us) client_negotiation.cc:240] Negotiation successful
0807 15:14:50.897537 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":119,"threads_started":1}
I20260807 15:14:50.898147 22142 heartbeater.cc:347] Connected to a master server at 127.29.194.62:42465
I20260807 15:14:50.898242 22142 heartbeater.cc:464] Registering TS with master...
I20260807 15:14:50.898396 22142 heartbeater.cc:511] Master 127.29.194.62:42465 requested a full tablet report, sending...
I20260807 15:14:50.898918 21659 ts_manager.cc:194] Registered new tserver with Master: 732c0e3bda8644778e6306e1aa2eb6d0 (127.29.194.2:37205)
I20260807 15:14:50.899513 21659 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.194.2@KRBTEST.COM'} at 127.29.194.2:55093
I20260807 15:14:50.900400 22142 heartbeater.cc:503] Master 127.29.194.62:42465 was elected leader, sending a full tablet report...
Aug 07 15:14:50 dist-test-slave-42z9 krb5kdc[20839](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115690, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.194.62@KRBTEST.COM for kudu/127.29.194.2@KRBTEST.COM
I20260807 15:14:50.907800 21955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:50.899222 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.2:37205 (local address 127.0.0.1:55512)
0807 15:14:50.899641 (+ 419us) negotiation.cc:107] Waiting for socket to connect
0807 15:14:50.899652 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 15:14:50.899704 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:14:50.899871 (+ 167us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:14:50.899872 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:14:50.900040 (+ 168us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:14:50.900091 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:14:50.900093 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:14:50.901482 (+ 1389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:14:50.901484 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:14:50.902117 (+ 633us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:14:50.902120 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:14:50.902812 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:14:50.902813 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:14:50.902970 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:50.903485 (+ 515us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:14:50.903489 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:14:50.904770 (+ 1281us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:14:50.906626 (+ 1856us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:14:50.906627 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:14:50.906628 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:14:50.906889 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:14:50.907133 (+ 244us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:14:50.907133 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:14:50.907134 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:14:50.907194 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:14:50.907458 (+ 264us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:14:50.907459 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:14:50.907511 (+ 52us) client_negotiation.cc:769] Sending connection context
0807 15:14:50.907577 (+ 66us) client_negotiation.cc:240] Negotiation successful
0807 15:14:50.907691 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":403}
I20260807 15:14:50.907852 22149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:50.899382 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55512 (local address 127.29.194.2:37205)
0807 15:14:50.899622 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0807 15:14:50.899627 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:14:50.899696 (+ 69us) server_negotiation.cc:407] Connection header received
0807 15:14:50.899760 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:14:50.899762 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:14:50.899780 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:14:50.899863 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:14:50.900193 (+ 330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:50.901241 (+ 1048us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:50.902204 (+ 963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:50.902669 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:50.902751 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:50.904895 (+ 2144us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:14:50.904906 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:14:50.904906 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:14:50.904915 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:14:50.906544 (+ 1629us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:14:50.906998 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:14:50.907001 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:14:50.907001 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:14:50.907071 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:14:50.907239 (+ 168us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:14:50.907240 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:14:50.907240 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:14:50.907399 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:14:50.907466 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0807 15:14:50.907674 (+ 208us) server_negotiation.cc:299] Negotiation successful
0807 15:14:50.907769 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":151,"thread_start_us":102,"threads_started":1}
I20260807 15:14:50.908258 22045 tablet_service.cc:1514] Processing DeleteTablet for tablet f6dc8f3e065e427086299e5878b671b5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:14:30 UTC) from {username='kudu', principal='kudu/127.29.194.62@KRBTEST.COM'} at 127.0.0.1:55512
I20260807 15:14:50.908950 22150 tablet_replica.cc:330] stopping tablet replica
I20260807 15:14:50.909056 22150 raft_consensus.cc:2229] T f6dc8f3e065e427086299e5878b671b5 P 732c0e3bda8644778e6306e1aa2eb6d0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 15:14:50.909129 22150 raft_consensus.cc:2258] T f6dc8f3e065e427086299e5878b671b5 P 732c0e3bda8644778e6306e1aa2eb6d0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:14:50.909524 22150 ts_tablet_manager.cc:1905] T f6dc8f3e065e427086299e5878b671b5 P 732c0e3bda8644778e6306e1aa2eb6d0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:14:50.911549 22150 ts_tablet_manager.cc:1918] T f6dc8f3e065e427086299e5878b671b5 P 732c0e3bda8644778e6306e1aa2eb6d0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 15:14:50.911623 22150 log.cc:1196] T f6dc8f3e065e427086299e5878b671b5 P 732c0e3bda8644778e6306e1aa2eb6d0: Deleting WAL directory at /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115480851507-30472-0/minicluster-data/ts-1/wal/wals/f6dc8f3e065e427086299e5878b671b5
I20260807 15:14:50.911835 22150 ts_tablet_manager.cc:1939] T f6dc8f3e065e427086299e5878b671b5 P 732c0e3bda8644778e6306e1aa2eb6d0: Deleting consensus metadata
I20260807 15:14:50.912423 21638 catalog_manager.cc:4841] TS 732c0e3bda8644778e6306e1aa2eb6d0 (127.29.194.2:37205): tablet f6dc8f3e065e427086299e5878b671b5 (table test-table [id=115c5af53e0c426fbac39efc831e81f8]) successfully deleted
W20260807 15:14:50.981897 22147 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:14:50.982149 22147 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:14:50.982182 22147 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:14:50.984009 22147 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:14:50.984067 22147 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:14:50.984095 22147 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.194.3
I20260807 15:14:50.985847 22147 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.194.20:40443
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115480851507-30472-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115480851507-30472-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.3.keytab
--principal=kudu/127.29.194.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.194.3:34775
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115480851507-30472-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.194.3
--webserver_port=40145
--tserver_master_addrs=127.29.194.62:42465
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22147
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.194.3
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115480851507-30472-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:14:50.986094 22147 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:14:50.986349 22147 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:14:50.991782 22158 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:14:50.991782 22159 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:14:50.991786 22161 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:14:50.992074 22147 server_base.cc:1034] running on GCE node
I20260807 15:14:50.992340 22147 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:14:50.992617 22147 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:14:50.993866 22147 hybrid_clock.cc:648] HybridClock initialized: now 1786115690993827 us; error 38 us; skew 500 ppm
Aug 07 15:14:50 dist-test-slave-42z9 krb5kdc[20839](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115690, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.194.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:14:50.997073 22147 init.cc:377] Logged in from keytab as kudu/127.29.194.3@KRBTEST.COM (short username kudu)
I20260807 15:14:50.997666 22147 webserver.cc:460] Webserver started at http://127.29.194.3:40145/ using document root <none> and password file <none>
I20260807 15:14:50.997889 22147 fs_manager.cc:359] Metadata directory not provided
I20260807 15:14:50.997956 22147 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:14:50.999413 22147 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:14:51.000290 22168 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:14:51.000574 22147 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:14:51.000690 22147 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115480851507-30472-0/minicluster-data/ts-2/data,/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115480851507-30472-0/minicluster-data/ts-2/wal
uuid: "d76b0cdebcc44295a6d2a7049ef4b030"
format_stamp: "Formatted at 2026-08-07 15:14:30 on dist-test-slave-42z9"
server_key: "e0006eb0c6c241ecdc04fe98b9047bc8"
server_key_iv: "45699283dc90bfe884ea33d509e9b9b3"
server_key_version: "encryptionkey@0"
I20260807 15:14:51.001072 22147 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115480851507-30472-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115480851507-30472-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115480851507-30472-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:14:51.011819 22147 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:14:51.014086 22147 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:14:51.014238 22147 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:14:51.014490 22147 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:14:51.014833 22147 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:14:51.014869 22147 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:14:51.014899 22147 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:14:51.015041 22147 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:14:51.024016 22147 rpc_server.cc:307] RPC server started. Bound to: 127.29.194.3:34775
I20260807 15:14:51.024060 22318 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.194.3:34775 every 8 connection(s)
I20260807 15:14:51.024428 22147 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115480851507-30472-0/minicluster-data/ts-2/data/info.pb
I20260807 15:14:51.027027 30472 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7fZSIL/build/release/bin/kudu as pid 22147
Aug 07 15:14:51 dist-test-slave-42z9 krb5kdc[20839](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115690, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.194.3@KRBTEST.COM for kudu/127.29.194.62@KRBTEST.COM
I20260807 15:14:51.035706 22328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:51.027426 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35372 (local address 127.29.194.62:42465)
0807 15:14:51.027552 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0807 15:14:51.027554 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:14:51.027624 (+ 70us) server_negotiation.cc:407] Connection header received
0807 15:14:51.027658 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:14:51.027658 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:14:51.027669 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:14:51.027736 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:14:51.028496 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:51.029047 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:51.030174 (+ 1127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:51.030397 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:51.030459 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:51.031844 (+ 1385us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:14:51.031846 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:14:51.031846 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:14:51.031854 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:14:51.034050 (+ 2196us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:14:51.034676 (+ 626us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:14:51.034677 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:14:51.034678 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:14:51.034780 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:14:51.035064 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:14:51.035064 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:14:51.035065 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:14:51.035278 (+ 213us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:14:51.035420 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0807 15:14:51.035546 (+ 126us) server_negotiation.cc:299] Negotiation successful
0807 15:14:51.035621 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":32,"server-negotiator.queue_time_us":109,"thread_start_us":57,"threads_started":1}
I20260807 15:14:51.035887 21950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:51.025186 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.194.3:51489 (local address 127.29.194.62:42465)
0807 15:14:51.025279 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0807 15:14:51.025281 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:14:51.025468 (+ 187us) server_negotiation.cc:407] Connection header received
0807 15:14:51.025957 (+ 489us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:14:51.025958 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:14:51.025968 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:14:51.026020 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:14:51.027042 (+ 1022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:51.027682 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:51.028940 (+ 1258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:51.029212 (+ 272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:51.029264 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:51.031844 (+ 2580us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:14:51.031846 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:14:51.031846 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:14:51.031855 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:14:51.034006 (+ 2151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:14:51.034695 (+ 689us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:14:51.034696 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:14:51.034696 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:14:51.034806 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:14:51.035128 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:14:51.035128 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:14:51.035129 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:14:51.035287 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:14:51.035363 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0807 15:14:51.035731 (+ 368us) server_negotiation.cc:299] Negotiation successful
0807 15:14:51.035812 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260807 15:14:51.036201 22324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:51.025163 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:42465 (local address 127.29.194.3:51489)
0807 15:14:51.025312 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0807 15:14:51.025349 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0807 15:14:51.025873 (+ 524us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:14:51.026083 (+ 210us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:14:51.026086 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:14:51.026673 (+ 587us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:14:51.026929 (+ 256us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:14:51.026935 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:14:51.027774 (+ 839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:14:51.027775 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:14:51.028839 (+ 1064us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:14:51.028842 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:14:51.029373 (+ 531us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:14:51.029374 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:14:51.029586 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:51.030127 (+ 541us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:14:51.030137 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:14:51.031721 (+ 1584us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:14:51.034119 (+ 2398us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:14:51.034132 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:14:51.034135 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:14:51.034605 (+ 470us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:14:51.034893 (+ 288us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:14:51.034894 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:14:51.034894 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:14:51.035069 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:14:51.035371 (+ 302us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:14:51.035380 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:14:51.035584 (+ 204us) client_negotiation.cc:769] Sending connection context
0807 15:14:51.035702 (+ 118us) client_negotiation.cc:240] Negotiation successful
0807 15:14:51.035886 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":84,"threads_started":1}
I20260807 15:14:51.036711 22319 heartbeater.cc:347] Connected to a master server at 127.29.194.62:42465
I20260807 15:14:51.036810 22319 heartbeater.cc:464] Registering TS with master...
I20260807 15:14:51.037009 22319 heartbeater.cc:511] Master 127.29.194.62:42465 requested a full tablet report, sending...
I20260807 15:14:51.037374 21659 ts_manager.cc:194] Registered new tserver with Master: d76b0cdebcc44295a6d2a7049ef4b030 (127.29.194.3:34775)
I20260807 15:14:51.037775 21659 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.194.3@KRBTEST.COM'} at 127.29.194.3:51489
I20260807 15:14:51.047712 30472 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 15:14:51.051404 22328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:51.048147 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35374 (local address 127.29.194.62:42465)
0807 15:14:51.048236 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0807 15:14:51.048239 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:14:51.048459 (+ 220us) server_negotiation.cc:407] Connection header received
0807 15:14:51.048579 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:14:51.048580 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:14:51.048643 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:14:51.048725 (+ 82us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 15:14:51.048939 (+ 214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:51.049597 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:51.050625 (+ 1028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:51.050870 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:51.050934 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:51.051096 (+ 162us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 15:14:51.051156 (+ 60us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0807 15:14:51.051325 (+ 169us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:35374: token expired
Metrics: {"server-negotiator.queue_time_us":34}
W20260807 15:14:51.051472 22328 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:35374: token expired
W20260807 15:14:51.051647 21568 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.29.194.62:42465)
I20260807 15:14:51.057929 22328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:51.051890 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35376 (local address 127.29.194.62:42465)
0807 15:14:51.051944 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0807 15:14:51.051946 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:14:51.051954 (+ 8us) server_negotiation.cc:407] Connection header received
0807 15:14:51.051977 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:14:51.051977 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:14:51.051986 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:14:51.052039 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:14:51.052551 (+ 512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:51.053030 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:51.053722 (+ 692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:51.053902 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:51.053954 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:51.055042 (+ 1088us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:14:51.055043 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:14:51.055043 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:14:51.055050 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:14:51.056445 (+ 1395us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:14:51.056835 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:14:51.056836 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:14:51.056836 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:14:51.056897 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:14:51.057112 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:14:51.057113 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:14:51.057113 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:14:51.057238 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:14:51.057329 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0807 15:14:51.057751 (+ 422us) server_negotiation.cc:299] Negotiation successful
0807 15:14:51.057852 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260807 15:14:51.059662 21659 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35376:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 15:14:51.061530 21659 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 15:14:51.063882 22045 tablet_service.cc:1467] Processing CreateTablet for tablet eb8b9a48cbcc4126aa34d2afe206581a (DEFAULT_TABLE table=test-table [id=60ae32ca313549d692efe92f75f07b8a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:14:51.064045 21836 tablet_service.cc:1467] Processing CreateTablet for tablet eb8b9a48cbcc4126aa34d2afe206581a (DEFAULT_TABLE table=test-table [id=60ae32ca313549d692efe92f75f07b8a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:14:51.064090 22045 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eb8b9a48cbcc4126aa34d2afe206581a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:14:51.064245 21836 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eb8b9a48cbcc4126aa34d2afe206581a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:14:51.065387 21987 tablet_bootstrap.cc:492] T eb8b9a48cbcc4126aa34d2afe206581a P 732c0e3bda8644778e6306e1aa2eb6d0: Bootstrap starting.
I20260807 15:14:51.065765 21987 tablet_bootstrap.cc:654] T eb8b9a48cbcc4126aa34d2afe206581a P 732c0e3bda8644778e6306e1aa2eb6d0: Neither blocks nor log segments found. Creating new log.
I20260807 15:14:51.065768 21785 tablet_bootstrap.cc:492] T eb8b9a48cbcc4126aa34d2afe206581a P 246195cca7b043878aaaf04013697524: Bootstrap starting.
I20260807 15:14:51.066074 21785 tablet_bootstrap.cc:654] T eb8b9a48cbcc4126aa34d2afe206581a P 246195cca7b043878aaaf04013697524: Neither blocks nor log segments found. Creating new log.
I20260807 15:14:51.066479 21987 tablet_bootstrap.cc:492] T eb8b9a48cbcc4126aa34d2afe206581a P 732c0e3bda8644778e6306e1aa2eb6d0: No bootstrap required, opened a new log
I20260807 15:14:51.066534 21987 ts_tablet_manager.cc:1397] T eb8b9a48cbcc4126aa34d2afe206581a P 732c0e3bda8644778e6306e1aa2eb6d0: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260807 15:14:51.066628 21785 tablet_bootstrap.cc:492] T eb8b9a48cbcc4126aa34d2afe206581a P 246195cca7b043878aaaf04013697524: No bootstrap required, opened a new log
I20260807 15:14:51.066679 21785 ts_tablet_manager.cc:1397] T eb8b9a48cbcc4126aa34d2afe206581a P 246195cca7b043878aaaf04013697524: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260807 15:14:51.066919 21987 raft_consensus.cc:348] T eb8b9a48cbcc4126aa34d2afe206581a P 732c0e3bda8644778e6306e1aa2eb6d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d76b0cdebcc44295a6d2a7049ef4b030" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 34775 } } peers { permanent_uuid: "246195cca7b043878aaaf04013697524" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 41283 } } peers { permanent_uuid: "732c0e3bda8644778e6306e1aa2eb6d0" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 37205 } }
I20260807 15:14:51.066989 21785 raft_consensus.cc:348] T eb8b9a48cbcc4126aa34d2afe206581a P 246195cca7b043878aaaf04013697524 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d76b0cdebcc44295a6d2a7049ef4b030" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 34775 } } peers { permanent_uuid: "246195cca7b043878aaaf04013697524" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 41283 } } peers { permanent_uuid: "732c0e3bda8644778e6306e1aa2eb6d0" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 37205 } }
I20260807 15:14:51.067015 21987 raft_consensus.cc:374] T eb8b9a48cbcc4126aa34d2afe206581a P 732c0e3bda8644778e6306e1aa2eb6d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:14:51.067050 21785 raft_consensus.cc:374] T eb8b9a48cbcc4126aa34d2afe206581a P 246195cca7b043878aaaf04013697524 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:14:51.067078 21987 raft_consensus.cc:729] T eb8b9a48cbcc4126aa34d2afe206581a P 732c0e3bda8644778e6306e1aa2eb6d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 732c0e3bda8644778e6306e1aa2eb6d0, State: Initialized, Role: FOLLOWER
I20260807 15:14:51.067090 21785 raft_consensus.cc:729] T eb8b9a48cbcc4126aa34d2afe206581a P 246195cca7b043878aaaf04013697524 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 246195cca7b043878aaaf04013697524, State: Initialized, Role: FOLLOWER
I20260807 15:14:51.067130 21785 consensus_queue.cc:260] T eb8b9a48cbcc4126aa34d2afe206581a P 246195cca7b043878aaaf04013697524 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d76b0cdebcc44295a6d2a7049ef4b030" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 34775 } } peers { permanent_uuid: "246195cca7b043878aaaf04013697524" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 41283 } } peers { permanent_uuid: "732c0e3bda8644778e6306e1aa2eb6d0" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 37205 } }
I20260807 15:14:51.067128 21987 consensus_queue.cc:260] T eb8b9a48cbcc4126aa34d2afe206581a P 732c0e3bda8644778e6306e1aa2eb6d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d76b0cdebcc44295a6d2a7049ef4b030" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 34775 } } peers { permanent_uuid: "246195cca7b043878aaaf04013697524" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 41283 } } peers { permanent_uuid: "732c0e3bda8644778e6306e1aa2eb6d0" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 37205 } }
I20260807 15:14:51.067219 21785 ts_tablet_manager.cc:1428] T eb8b9a48cbcc4126aa34d2afe206581a P 246195cca7b043878aaaf04013697524: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260807 15:14:51.067219 21987 ts_tablet_manager.cc:1428] T eb8b9a48cbcc4126aa34d2afe206581a P 732c0e3bda8644778e6306e1aa2eb6d0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260807 15:14:51.067749 21955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:51.063558 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.3:34775 (local address 127.0.0.1:51584)
0807 15:14:51.063608 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0807 15:14:51.063621 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 15:14:51.063673 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:14:51.064089 (+ 416us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:14:51.064090 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:14:51.064093 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:14:51.064147 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:14:51.064149 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:14:51.065510 (+ 1361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:14:51.065511 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:14:51.066657 (+ 1146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:14:51.066661 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:14:51.067486 (+ 825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:14:51.067487 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:14:51.067548 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:51.067574 (+ 26us) client_negotiation.cc:769] Sending connection context
0807 15:14:51.067630 (+ 56us) client_negotiation.cc:240] Negotiation successful
0807 15:14:51.067677 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30}
I20260807 15:14:51.068041 22330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:51.063682 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51584 (local address 127.29.194.3:34775)
0807 15:14:51.063892 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0807 15:14:51.063896 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:14:51.063908 (+ 12us) server_negotiation.cc:407] Connection header received
0807 15:14:51.063950 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:14:51.063952 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:14:51.063969 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:14:51.064223 (+ 254us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:14:51.064302 (+ 79us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:51.065360 (+ 1058us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:51.066763 (+ 1403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:51.067407 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:51.067490 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:51.067559 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0807 15:14:51.067911 (+ 352us) server_negotiation.cc:299] Negotiation successful
0807 15:14:51.067962 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":175,"thread_start_us":128,"threads_started":1}
I20260807 15:14:51.068727 22231 tablet_service.cc:1467] Processing CreateTablet for tablet eb8b9a48cbcc4126aa34d2afe206581a (DEFAULT_TABLE table=test-table [id=60ae32ca313549d692efe92f75f07b8a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:14:51.069092 22231 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eb8b9a48cbcc4126aa34d2afe206581a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:14:51.072000 22333 tablet_bootstrap.cc:492] T eb8b9a48cbcc4126aa34d2afe206581a P d76b0cdebcc44295a6d2a7049ef4b030: Bootstrap starting.
I20260807 15:14:51.072609 22333 tablet_bootstrap.cc:654] T eb8b9a48cbcc4126aa34d2afe206581a P d76b0cdebcc44295a6d2a7049ef4b030: Neither blocks nor log segments found. Creating new log.
I20260807 15:14:51.073094 22333 log.cc:824] T eb8b9a48cbcc4126aa34d2afe206581a P d76b0cdebcc44295a6d2a7049ef4b030: Log is configured to *not* fsync() on all Append() calls
I20260807 15:14:51.074295 22333 tablet_bootstrap.cc:492] T eb8b9a48cbcc4126aa34d2afe206581a P d76b0cdebcc44295a6d2a7049ef4b030: No bootstrap required, opened a new log
I20260807 15:14:51.074366 22333 ts_tablet_manager.cc:1397] T eb8b9a48cbcc4126aa34d2afe206581a P d76b0cdebcc44295a6d2a7049ef4b030: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 15:14:51.075740 22333 raft_consensus.cc:348] T eb8b9a48cbcc4126aa34d2afe206581a P d76b0cdebcc44295a6d2a7049ef4b030 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d76b0cdebcc44295a6d2a7049ef4b030" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 34775 } } peers { permanent_uuid: "246195cca7b043878aaaf04013697524" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 41283 } } peers { permanent_uuid: "732c0e3bda8644778e6306e1aa2eb6d0" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 37205 } }
I20260807 15:14:51.075891 22333 raft_consensus.cc:374] T eb8b9a48cbcc4126aa34d2afe206581a P d76b0cdebcc44295a6d2a7049ef4b030 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:14:51.075923 22333 raft_consensus.cc:729] T eb8b9a48cbcc4126aa34d2afe206581a P d76b0cdebcc44295a6d2a7049ef4b030 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d76b0cdebcc44295a6d2a7049ef4b030, State: Initialized, Role: FOLLOWER
I20260807 15:14:51.076049 22333 consensus_queue.cc:260] T eb8b9a48cbcc4126aa34d2afe206581a P d76b0cdebcc44295a6d2a7049ef4b030 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d76b0cdebcc44295a6d2a7049ef4b030" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 34775 } } peers { permanent_uuid: "246195cca7b043878aaaf04013697524" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 41283 } } peers { permanent_uuid: "732c0e3bda8644778e6306e1aa2eb6d0" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 37205 } }
I20260807 15:14:51.076336 22333 ts_tablet_manager.cc:1428] T eb8b9a48cbcc4126aa34d2afe206581a P d76b0cdebcc44295a6d2a7049ef4b030: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260807 15:14:51.076406 22319 heartbeater.cc:503] Master 127.29.194.62:42465 was elected leader, sending a full tablet report...
W20260807 15:14:51.137836 22143 tablet.cc:2369] T eb8b9a48cbcc4126aa34d2afe206581a P 732c0e3bda8644778e6306e1aa2eb6d0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 15:14:51.195089 21953 raft_consensus.cc:482] T eb8b9a48cbcc4126aa34d2afe206581a P 246195cca7b043878aaaf04013697524 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 15:14:51.195300 21953 raft_consensus.cc:504] T eb8b9a48cbcc4126aa34d2afe206581a P 246195cca7b043878aaaf04013697524 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d76b0cdebcc44295a6d2a7049ef4b030" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 34775 } } peers { permanent_uuid: "246195cca7b043878aaaf04013697524" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 41283 } } peers { permanent_uuid: "732c0e3bda8644778e6306e1aa2eb6d0" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 37205 } }
I20260807 15:14:51.195833 21953 leader_election.cc:288] T eb8b9a48cbcc4126aa34d2afe206581a P 246195cca7b043878aaaf04013697524 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d76b0cdebcc44295a6d2a7049ef4b030 (127.29.194.3:34775), 732c0e3bda8644778e6306e1aa2eb6d0 (127.29.194.2:37205)
I20260807 15:14:51.199229 22330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:51.196318 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.194.1:37293 (local address 127.29.194.3:34775)
0807 15:14:51.196393 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0807 15:14:51.196396 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:14:51.196408 (+ 12us) server_negotiation.cc:407] Connection header received
0807 15:14:51.196441 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:14:51.196442 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:14:51.196451 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:14:51.196536 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:14:51.196715 (+ 179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:51.197320 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:51.198462 (+ 1142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:51.198849 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:51.198908 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:51.198916 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0807 15:14:51.199104 (+ 188us) server_negotiation.cc:299] Negotiation successful
0807 15:14:51.199147 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260807 15:14:51.199252 22149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:51.196447 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.194.1:35401 (local address 127.29.194.2:37205)
0807 15:14:51.196497 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0807 15:14:51.196499 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:14:51.196634 (+ 135us) server_negotiation.cc:407] Connection header received
0807 15:14:51.196668 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:14:51.196669 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:14:51.196697 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:14:51.196753 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:14:51.196941 (+ 188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:51.197468 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:51.198449 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:51.198876 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:51.198958 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:51.199035 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0807 15:14:51.199132 (+ 97us) server_negotiation.cc:299] Negotiation successful
0807 15:14:51.199181 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260807 15:14:51.199270 21949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:51.196160 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.3:34775 (local address 127.29.194.1:37293)
0807 15:14:51.196225 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0807 15:14:51.196238 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 15:14:51.196320 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:14:51.196555 (+ 235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:14:51.196556 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:14:51.196558 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:14:51.196633 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:14:51.196635 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:14:51.197412 (+ 777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:14:51.197413 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:14:51.198359 (+ 946us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:14:51.198363 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:14:51.198932 (+ 569us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:14:51.198933 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:14:51.199006 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:51.199035 (+ 29us) client_negotiation.cc:769] Sending connection context
0807 15:14:51.199089 (+ 54us) client_negotiation.cc:240] Negotiation successful
0807 15:14:51.199143 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35,"mutex_wait_us":22}
I20260807 15:14:51.199275 22336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:51.196317 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.2:37205 (local address 127.29.194.1:35401)
0807 15:14:51.196557 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0807 15:14:51.196567 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 15:14:51.196629 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:14:51.196770 (+ 141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:14:51.196771 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:14:51.196773 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:14:51.196849 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:14:51.196850 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:14:51.197549 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:14:51.197550 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:14:51.198361 (+ 811us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:14:51.198364 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:14:51.198990 (+ 626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:14:51.198991 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:14:51.199056 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:51.199059 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 15:14:51.199108 (+ 49us) client_negotiation.cc:240] Negotiation successful
0807 15:14:51.199146 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":218,"mutex_wait_us":3,"thread_start_us":142,"threads_started":1}
I20260807 15:14:51.199695 22072 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eb8b9a48cbcc4126aa34d2afe206581a" candidate_uuid: "246195cca7b043878aaaf04013697524" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "732c0e3bda8644778e6306e1aa2eb6d0" is_pre_election: true
I20260807 15:14:51.199725 22257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eb8b9a48cbcc4126aa34d2afe206581a" candidate_uuid: "246195cca7b043878aaaf04013697524" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d76b0cdebcc44295a6d2a7049ef4b030" is_pre_election: true
I20260807 15:14:51.199863 22072 raft_consensus.cc:2454] T eb8b9a48cbcc4126aa34d2afe206581a P 732c0e3bda8644778e6306e1aa2eb6d0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 246195cca7b043878aaaf04013697524 in term 0.
I20260807 15:14:51.199887 22257 raft_consensus.cc:2454] T eb8b9a48cbcc4126aa34d2afe206581a P d76b0cdebcc44295a6d2a7049ef4b030 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 246195cca7b043878aaaf04013697524 in term 0.
I20260807 15:14:51.200091 21778 leader_election.cc:302] T eb8b9a48cbcc4126aa34d2afe206581a P 246195cca7b043878aaaf04013697524 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 246195cca7b043878aaaf04013697524, d76b0cdebcc44295a6d2a7049ef4b030; no voters:
I20260807 15:14:51.200229 21953 raft_consensus.cc:2781] T eb8b9a48cbcc4126aa34d2afe206581a P 246195cca7b043878aaaf04013697524 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 15:14:51.200292 21953 raft_consensus.cc:482] T eb8b9a48cbcc4126aa34d2afe206581a P 246195cca7b043878aaaf04013697524 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 15:14:51.200316 21953 raft_consensus.cc:3037] T eb8b9a48cbcc4126aa34d2afe206581a P 246195cca7b043878aaaf04013697524 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:14:51.201032 21953 raft_consensus.cc:504] T eb8b9a48cbcc4126aa34d2afe206581a P 246195cca7b043878aaaf04013697524 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d76b0cdebcc44295a6d2a7049ef4b030" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 34775 } } peers { permanent_uuid: "246195cca7b043878aaaf04013697524" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 41283 } } peers { permanent_uuid: "732c0e3bda8644778e6306e1aa2eb6d0" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 37205 } }
I20260807 15:14:51.201167 21953 leader_election.cc:288] T eb8b9a48cbcc4126aa34d2afe206581a P 246195cca7b043878aaaf04013697524 [CANDIDATE]: Term 1 election: Requested vote from peers d76b0cdebcc44295a6d2a7049ef4b030 (127.29.194.3:34775), 732c0e3bda8644778e6306e1aa2eb6d0 (127.29.194.2:37205)
I20260807 15:14:51.201375 22257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eb8b9a48cbcc4126aa34d2afe206581a" candidate_uuid: "246195cca7b043878aaaf04013697524" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d76b0cdebcc44295a6d2a7049ef4b030"
I20260807 15:14:51.201375 22072 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eb8b9a48cbcc4126aa34d2afe206581a" candidate_uuid: "246195cca7b043878aaaf04013697524" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "732c0e3bda8644778e6306e1aa2eb6d0"
I20260807 15:14:51.201493 22072 raft_consensus.cc:3037] T eb8b9a48cbcc4126aa34d2afe206581a P 732c0e3bda8644778e6306e1aa2eb6d0 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:14:51.201493 22257 raft_consensus.cc:3037] T eb8b9a48cbcc4126aa34d2afe206581a P d76b0cdebcc44295a6d2a7049ef4b030 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:14:51.202217 22257 raft_consensus.cc:2454] T eb8b9a48cbcc4126aa34d2afe206581a P d76b0cdebcc44295a6d2a7049ef4b030 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 246195cca7b043878aaaf04013697524 in term 1.
I20260807 15:14:51.202278 22072 raft_consensus.cc:2454] T eb8b9a48cbcc4126aa34d2afe206581a P 732c0e3bda8644778e6306e1aa2eb6d0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 246195cca7b043878aaaf04013697524 in term 1.
I20260807 15:14:51.202397 21778 leader_election.cc:302] T eb8b9a48cbcc4126aa34d2afe206581a P 246195cca7b043878aaaf04013697524 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 246195cca7b043878aaaf04013697524, d76b0cdebcc44295a6d2a7049ef4b030; no voters:
I20260807 15:14:51.202505 21953 raft_consensus.cc:2781] T eb8b9a48cbcc4126aa34d2afe206581a P 246195cca7b043878aaaf04013697524 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:14:51.202631 21953 raft_consensus.cc:686] T eb8b9a48cbcc4126aa34d2afe206581a P 246195cca7b043878aaaf04013697524 [term 1 LEADER]: Becoming Leader. State: Replica: 246195cca7b043878aaaf04013697524, State: Running, Role: LEADER
I20260807 15:14:51.202708 21953 consensus_queue.cc:237] T eb8b9a48cbcc4126aa34d2afe206581a P 246195cca7b043878aaaf04013697524 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d76b0cdebcc44295a6d2a7049ef4b030" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 34775 } } peers { permanent_uuid: "246195cca7b043878aaaf04013697524" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 41283 } } peers { permanent_uuid: "732c0e3bda8644778e6306e1aa2eb6d0" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 37205 } }
I20260807 15:14:51.203784 21658 catalog_manager.cc:5495] T eb8b9a48cbcc4126aa34d2afe206581a P 246195cca7b043878aaaf04013697524 reported cstate change: term changed from 0 to 1, leader changed from <none> to 246195cca7b043878aaaf04013697524 (127.29.194.1). New cstate: current_term: 1 leader_uuid: "246195cca7b043878aaaf04013697524" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d76b0cdebcc44295a6d2a7049ef4b030" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 34775 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "246195cca7b043878aaaf04013697524" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 41283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "732c0e3bda8644778e6306e1aa2eb6d0" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 37205 } health_report { overall_health: UNKNOWN } } }
I20260807 15:14:51.217780 21956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:51.215258 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40148 (local address 127.29.194.1:41283)
0807 15:14:51.215337 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0807 15:14:51.215339 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:14:51.215348 (+ 9us) server_negotiation.cc:407] Connection header received
0807 15:14:51.215394 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:14:51.215395 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:14:51.215415 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:14:51.215478 (+ 63us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 15:14:51.215622 (+ 144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:51.216215 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:51.216981 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:51.217196 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:51.217252 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:51.217399 (+ 147us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 15:14:51.217552 (+ 153us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 15:14:51.217618 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0807 15:14:51.217671 (+ 53us) server_negotiation.cc:299] Negotiation successful
0807 15:14:51.217695 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260807 15:14:51.219890 22257 raft_consensus.cc:1261] T eb8b9a48cbcc4126aa34d2afe206581a P d76b0cdebcc44295a6d2a7049ef4b030 [term 1 FOLLOWER]: Refusing update from remote peer 246195cca7b043878aaaf04013697524: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:14:51.219890 22072 raft_consensus.cc:1261] T eb8b9a48cbcc4126aa34d2afe206581a P 732c0e3bda8644778e6306e1aa2eb6d0 [term 1 FOLLOWER]: Refusing update from remote peer 246195cca7b043878aaaf04013697524: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:14:51.220293 22337 consensus_queue.cc:1035] T eb8b9a48cbcc4126aa34d2afe206581a P 246195cca7b043878aaaf04013697524 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d76b0cdebcc44295a6d2a7049ef4b030" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 34775 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:14:51.220463 21953 consensus_queue.cc:1035] T eb8b9a48cbcc4126aa34d2afe206581a P 246195cca7b043878aaaf04013697524 [LEADER]: Connected to new peer: Peer: permanent_uuid: "732c0e3bda8644778e6306e1aa2eb6d0" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 37205 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:14:51.222658 22340 mvcc.cc:205] Tried to move back new op lower bound from 7315929871233486848 to 7315929871166795776. Current Snapshot: MvccSnapshot[applied={T|T < 7315929871233486848}]
I20260807 15:14:51.222863 22343 mvcc.cc:205] Tried to move back new op lower bound from 7315929871233486848 to 7315929871166795776. Current Snapshot: MvccSnapshot[applied={T|T < 7315929871233486848}]
I20260807 15:14:51.223127 22342 mvcc.cc:205] Tried to move back new op lower bound from 7315929871233486848 to 7315929871166795776. Current Snapshot: MvccSnapshot[applied={T|T < 7315929871233486848}]
I20260807 15:14:51.226033 21659 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35376:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 15:14:51.226141 21659 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35376:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 15:14:51.226945 21659 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4e95b6a1e91e4fc0b95dbb205a0d0d4a: Sending DeleteTablet for 3 replicas of tablet eb8b9a48cbcc4126aa34d2afe206581a
I20260807 15:14:51.227263 22231 tablet_service.cc:1514] Processing DeleteTablet for tablet eb8b9a48cbcc4126aa34d2afe206581a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:14:51 UTC) from {username='kudu', principal='kudu/127.29.194.62@KRBTEST.COM'} at 127.0.0.1:51584
I20260807 15:14:51.227295 30472 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7fZSIL/build/release/bin/kudu with pid 21715
I20260807 15:14:51.227306 22045 tablet_service.cc:1514] Processing DeleteTablet for tablet eb8b9a48cbcc4126aa34d2afe206581a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:14:51 UTC) from {username='kudu', principal='kudu/127.29.194.62@KRBTEST.COM'} at 127.0.0.1:55512
I20260807 15:14:51.227398 22150 tablet_replica.cc:330] T eb8b9a48cbcc4126aa34d2afe206581a P 732c0e3bda8644778e6306e1aa2eb6d0: stopping tablet replica
I20260807 15:14:51.227445 22150 raft_consensus.cc:2229] T eb8b9a48cbcc4126aa34d2afe206581a P 732c0e3bda8644778e6306e1aa2eb6d0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 15:14:51.227509 22150 raft_consensus.cc:2258] T eb8b9a48cbcc4126aa34d2afe206581a P 732c0e3bda8644778e6306e1aa2eb6d0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:14:51.233062 22150 ts_tablet_manager.cc:1905] T eb8b9a48cbcc4126aa34d2afe206581a P 732c0e3bda8644778e6306e1aa2eb6d0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:14:51.236944 22150 ts_tablet_manager.cc:1918] T eb8b9a48cbcc4126aa34d2afe206581a P 732c0e3bda8644778e6306e1aa2eb6d0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 15:14:51.237037 22150 log.cc:1196] T eb8b9a48cbcc4126aa34d2afe206581a P 732c0e3bda8644778e6306e1aa2eb6d0: Deleting WAL directory at /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115480851507-30472-0/minicluster-data/ts-1/wal/wals/eb8b9a48cbcc4126aa34d2afe206581a
I20260807 15:14:51.237282 22150 ts_tablet_manager.cc:1939] T eb8b9a48cbcc4126aa34d2afe206581a P 732c0e3bda8644778e6306e1aa2eb6d0: Deleting consensus metadata
I20260807 15:14:51.239367 21638 catalog_manager.cc:4841] TS 732c0e3bda8644778e6306e1aa2eb6d0 (127.29.194.2:37205): tablet eb8b9a48cbcc4126aa34d2afe206581a (table test-table [id=60ae32ca313549d692efe92f75f07b8a]) successfully deleted
I20260807 15:14:51.244580 22351 tablet_replica.cc:330] T eb8b9a48cbcc4126aa34d2afe206581a P d76b0cdebcc44295a6d2a7049ef4b030: stopping tablet replica
I20260807 15:14:51.244769 22351 raft_consensus.cc:2229] T eb8b9a48cbcc4126aa34d2afe206581a P d76b0cdebcc44295a6d2a7049ef4b030 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 15:14:51.244900 22351 raft_consensus.cc:2258] T eb8b9a48cbcc4126aa34d2afe206581a P d76b0cdebcc44295a6d2a7049ef4b030 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:14:51.246737 22351 ts_tablet_manager.cc:1905] T eb8b9a48cbcc4126aa34d2afe206581a P d76b0cdebcc44295a6d2a7049ef4b030: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:14:51.248616 22351 ts_tablet_manager.cc:1918] T eb8b9a48cbcc4126aa34d2afe206581a P d76b0cdebcc44295a6d2a7049ef4b030: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 15:14:51.248716 22351 log.cc:1196] T eb8b9a48cbcc4126aa34d2afe206581a P d76b0cdebcc44295a6d2a7049ef4b030: Deleting WAL directory at /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115480851507-30472-0/minicluster-data/ts-2/wal/wals/eb8b9a48cbcc4126aa34d2afe206581a
I20260807 15:14:51.248989 22351 ts_tablet_manager.cc:1939] T eb8b9a48cbcc4126aa34d2afe206581a P d76b0cdebcc44295a6d2a7049ef4b030: Deleting consensus metadata
I20260807 15:14:51.249529 21638 catalog_manager.cc:4841] TS d76b0cdebcc44295a6d2a7049ef4b030 (127.29.194.3:34775): tablet eb8b9a48cbcc4126aa34d2afe206581a (table test-table [id=60ae32ca313549d692efe92f75f07b8a]) successfully deleted
W20260807 15:14:51.250957 21638 connection.cc:535] client connection to 127.29.194.1:41283 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260807 15:14:51.251093 21638 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260807 15:14:51.251575 21955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:51.251452 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.1:41283 (local address 127.0.0.1:40164)
0807 15:14:51.251505 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0807 15:14:51.251544 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.194.1:41283: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260807 15:14:51.251679 30472 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7fZSIL/build/release/bin/kudu with pid 21951
W20260807 15:14:51.257828 21638 catalog_manager.cc:4584] TS 246195cca7b043878aaaf04013697524 (127.29.194.1:41283): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet eb8b9a48cbcc4126aa34d2afe206581a: Network error: Client connection negotiation failed: client connection to 127.29.194.1:41283: connect: Connection refused (error 111)
I20260807 15:14:51.263756 30472 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7fZSIL/build/release/bin/kudu with pid 22147
I20260807 15:14:51.281713 30472 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7fZSIL/build/release/bin/kudu with pid 21614
I20260807 15:15:11.303813 30472 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7fZSIL/build/release/bin/kudu
/tmp/dist-test-task7fZSIL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115480851507-30472-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115480851507-30472-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115480851507-30472-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115480851507-30472-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.194.62:42465
--webserver_interface=127.29.194.62
--webserver_port=41213
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.62.keytab
--principal=kudu/127.29.194.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.194.20:40443
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.194.62:42465
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115480851507-30472-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115480851507-30472-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115480851507-30472-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:15:11.392491 22354 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:15:11.392736 22354 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:15:11.392761 22354 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:15:11.394070 22354 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260807 15:15:11.394126 22354 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260807 15:15:11.394140 22354 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260807 15:15:11.394150 22354 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:15:11.394162 22354 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:15:11.394178 22354 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:15:11.394202 22354 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:15:11.394225 22354 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:15:11.395922 22354 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.194.20:40443
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115480851507-30472-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115480851507-30472-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.29.194.62:42465
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.62.keytab
--principal=kudu/127.29.194.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.194.62:42465
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115480851507-30472-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.194.62
--webserver_port=41213
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22354
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115480851507-30472-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:15:11.396212 22354 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:15:11.396461 22354 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:15:11.402446 22360 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:15:11.402567 22354 server_base.cc:1034] running on GCE node
W20260807 15:15:11.402513 22363 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:15:11.402515 22359 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:15:11.403034 22354 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:15:11.403398 22354 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:15:11.404616 22354 hybrid_clock.cc:648] HybridClock initialized: now 1786115711404578 us; error 62 us; skew 500 ppm
Aug 07 15:15:11 dist-test-slave-42z9 krb5kdc[20839](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115711, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.194.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:15:11.412385 22354 init.cc:377] Logged in from keytab as kudu/127.29.194.62@KRBTEST.COM (short username kudu)
I20260807 15:15:11.412994 22354 webserver.cc:460] Webserver started at http://127.29.194.62:41213/ using document root <none> and password file <none>
I20260807 15:15:11.413251 22354 fs_manager.cc:359] Metadata directory not provided
I20260807 15:15:11.413305 22354 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:15:11.414820 22354 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:15:11.415604 22372 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:15:11.415827 22354 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:15:11.415930 22354 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115480851507-30472-0/minicluster-data/master-0/data,/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115480851507-30472-0/minicluster-data/master-0/wal
uuid: "4e95b6a1e91e4fc0b95dbb205a0d0d4a"
format_stamp: "Formatted at 2026-08-07 15:14:29 on dist-test-slave-42z9"
server_key: "964c398495f7750df66021b9acd44de6"
server_key_iv: "79b9c8fbbc418736323465fe6474d8fc"
server_key_version: "encryptionkey@0"
I20260807 15:15:11.416311 22354 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115480851507-30472-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115480851507-30472-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115480851507-30472-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:15:11.433696 22354 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:15:11.435942 22354 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:15:11.436105 22354 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:15:11.440575 22354 rpc_server.cc:307] RPC server started. Bound to: 127.29.194.62:42465
I20260807 15:15:11.440620 22458 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.194.62:42465 every 8 connection(s)
I20260807 15:15:11.440999 22354 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115480851507-30472-0/minicluster-data/master-0/data/info.pb
I20260807 15:15:11.441215 30472 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7fZSIL/build/release/bin/kudu as pid 22354
I20260807 15:15:11.441545 30472 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7fZSIL/build/release/bin/kudu
/tmp/dist-test-task7fZSIL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115480851507-30472-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115480851507-30472-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115480851507-30472-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115480851507-30472-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.194.1:41283
--local_ip_for_outbound_sockets=127.29.194.1
--tserver_master_addrs=127.29.194.62:42465
--webserver_port=40433
--webserver_interface=127.29.194.1
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.1.keytab
--principal=kudu/127.29.194.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.194.20:40443
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115480851507-30472-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115480851507-30472-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115480851507-30472-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 15:15:11.442631 22459 sys_catalog.cc:263] Verifying existing consensus state
I20260807 15:15:11.443346 22459 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4e95b6a1e91e4fc0b95dbb205a0d0d4a: Bootstrap starting.
I20260807 15:15:11.455101 22459 log.cc:824] T 00000000000000000000000000000000 P 4e95b6a1e91e4fc0b95dbb205a0d0d4a: Log is configured to *not* fsync() on all Append() calls
I20260807 15:15:11.458221 22459 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4e95b6a1e91e4fc0b95dbb205a0d0d4a: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 15:15:11.458609 22459 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4e95b6a1e91e4fc0b95dbb205a0d0d4a: Bootstrap complete.
I20260807 15:15:11.461933 22459 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4e95b6a1e91e4fc0b95dbb205a0d0d4a [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e95b6a1e91e4fc0b95dbb205a0d0d4a" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 42465 } }
I20260807 15:15:11.462280 22459 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4e95b6a1e91e4fc0b95dbb205a0d0d4a [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e95b6a1e91e4fc0b95dbb205a0d0d4a, State: Initialized, Role: FOLLOWER
I20260807 15:15:11.462472 22459 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4e95b6a1e91e4fc0b95dbb205a0d0d4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e95b6a1e91e4fc0b95dbb205a0d0d4a" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 42465 } }
I20260807 15:15:11.462560 22459 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4e95b6a1e91e4fc0b95dbb205a0d0d4a [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:15:11.462615 22459 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4e95b6a1e91e4fc0b95dbb205a0d0d4a [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:15:11.462662 22459 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4e95b6a1e91e4fc0b95dbb205a0d0d4a [term 2 FOLLOWER]: Advancing to term 3
I20260807 15:15:11.464023 22459 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4e95b6a1e91e4fc0b95dbb205a0d0d4a [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e95b6a1e91e4fc0b95dbb205a0d0d4a" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 42465 } }
I20260807 15:15:11.464146 22459 leader_election.cc:302] T 00000000000000000000000000000000 P 4e95b6a1e91e4fc0b95dbb205a0d0d4a [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4e95b6a1e91e4fc0b95dbb205a0d0d4a; no voters:
I20260807 15:15:11.464325 22459 leader_election.cc:288] T 00000000000000000000000000000000 P 4e95b6a1e91e4fc0b95dbb205a0d0d4a [CANDIDATE]: Term 3 election: Requested vote from peers
I20260807 15:15:11.464471 22464 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4e95b6a1e91e4fc0b95dbb205a0d0d4a [term 3 FOLLOWER]: Leader election won for term 3
I20260807 15:15:11.464586 22459 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4e95b6a1e91e4fc0b95dbb205a0d0d4a [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:15:11.464643 22464 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4e95b6a1e91e4fc0b95dbb205a0d0d4a [term 3 LEADER]: Becoming Leader. State: Replica: 4e95b6a1e91e4fc0b95dbb205a0d0d4a, State: Running, Role: LEADER
I20260807 15:15:11.464741 22464 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4e95b6a1e91e4fc0b95dbb205a0d0d4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e95b6a1e91e4fc0b95dbb205a0d0d4a" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 42465 } }
I20260807 15:15:11.465027 22465 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4e95b6a1e91e4fc0b95dbb205a0d0d4a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "4e95b6a1e91e4fc0b95dbb205a0d0d4a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e95b6a1e91e4fc0b95dbb205a0d0d4a" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 42465 } } }
I20260807 15:15:11.465037 22468 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4e95b6a1e91e4fc0b95dbb205a0d0d4a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4e95b6a1e91e4fc0b95dbb205a0d0d4a. Latest consensus state: current_term: 3 leader_uuid: "4e95b6a1e91e4fc0b95dbb205a0d0d4a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e95b6a1e91e4fc0b95dbb205a0d0d4a" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 42465 } } }
I20260807 15:15:11.465117 22465 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4e95b6a1e91e4fc0b95dbb205a0d0d4a [sys.catalog]: This master's current role is: LEADER
I20260807 15:15:11.465145 22468 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4e95b6a1e91e4fc0b95dbb205a0d0d4a [sys.catalog]: This master's current role is: LEADER
I20260807 15:15:11.465365 22474 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:15:11.466238 22474 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:15:11.466357 22474 catalog_manager.cc:1259] Loaded cluster ID: e054a19df1ae4e99a1ba8f154ecc4884
I20260807 15:15:11.466390 22474 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:15:11.467536 22474 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:15:11.467724 22474 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 4e95b6a1e91e4fc0b95dbb205a0d0d4a: Loaded TSK: 0
I20260807 15:15:11.467775 22474 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 4e95b6a1e91e4fc0b95dbb205a0d0d4a: Loaded TSK: 1
F20260807 15:15:11.468000 22474 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f04cb4b79ad google::LogMessage::Fail() at ??:0
@ 0x7f04cb4ba88a google::LogMessage::SendToLog() at ??:0
@ 0x7f04cb4b74de google::LogMessage::Flush() at ??:0
@ 0x7f04cb4b8a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f04cdc24104 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7f04cdc21d41 kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7f04d34111cd kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7f04d341148e kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7f04d3411594 _ZNSt17_Function_handlerIFN4kudu6StatusEvEZNS0_6master14CatalogManager24PrepareForLeadershipTaskEvEUlvE3_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x7f04d33d7c48 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7f04d33e1ad0 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f04cd3850d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f04cd37a954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f04ce1de6db start_thread at ??:0
@ 0x7f04c9fb561f clone at ??:0
W20260807 15:15:11.559643 22462 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:15:11.559975 22462 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:15:11.560019 22462 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:15:11.561848 22462 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:15:11.561918 22462 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:15:11.561960 22462 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.194.1
I20260807 15:15:11.564527 22462 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.194.20:40443
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115480851507-30472-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115480851507-30472-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.1.keytab
--principal=kudu/127.29.194.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.194.1:41283
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115480851507-30472-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.194.1
--webserver_port=40433
--tserver_master_addrs=127.29.194.62:42465
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22462
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.194.1
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115480851507-30472-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:15:11.564850 22462 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:15:11.565183 22462 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:15:11.570912 22515 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:15:11.571106 22514 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:15:11.570912 22517 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:15:11.571110 22462 server_base.cc:1034] running on GCE node
I20260807 15:15:11.571408 22462 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:15:11.571658 22462 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:15:11.572844 22462 hybrid_clock.cc:648] HybridClock initialized: now 1786115711572816 us; error 45 us; skew 500 ppm
Aug 07 15:15:11 dist-test-slave-42z9 krb5kdc[20839](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115711, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.194.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:15:11.575562 22462 init.cc:377] Logged in from keytab as kudu/127.29.194.1@KRBTEST.COM (short username kudu)
I20260807 15:15:11.576205 22462 webserver.cc:460] Webserver started at http://127.29.194.1:40433/ using document root <none> and password file <none>
I20260807 15:15:11.576499 22462 fs_manager.cc:359] Metadata directory not provided
I20260807 15:15:11.576570 22462 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:15:11.578459 22462 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.000s
I20260807 15:15:11.579293 22531 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:15:11.579507 22462 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:15:11.579594 22462 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115480851507-30472-0/minicluster-data/ts-0/data,/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115480851507-30472-0/minicluster-data/ts-0/wal
uuid: "246195cca7b043878aaaf04013697524"
format_stamp: "Formatted at 2026-08-07 15:14:29 on dist-test-slave-42z9"
server_key: "11e8656c00110a93fcf480a485a0826c"
server_key_iv: "4716fd9bfb72a0e363b1a5b3a1dc54f1"
server_key_version: "encryptionkey@0"
I20260807 15:15:11.579949 22462 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115480851507-30472-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115480851507-30472-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115480851507-30472-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:15:11.589946 22462 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:15:11.592518 22462 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:15:11.592650 22462 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:15:11.592891 22462 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:15:11.593351 22544 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260807 15:15:11.595458 22462 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260807 15:15:11.595520 22462 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260807 15:15:11.595558 22462 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260807 15:15:11.596232 22462 ts_tablet_manager.cc:610] Registered 1 tablets
I20260807 15:15:11.596267 22462 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260807 15:15:11.596361 22544 tablet_bootstrap.cc:492] T eb8b9a48cbcc4126aa34d2afe206581a P 246195cca7b043878aaaf04013697524: Bootstrap starting.
I20260807 15:15:11.604295 22462 rpc_server.cc:307] RPC server started. Bound to: 127.29.194.1:41283
I20260807 15:15:11.605115 22462 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115480851507-30472-0/minicluster-data/ts-0/data/info.pb
I20260807 15:15:11.607102 22698 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.194.1:41283 every 8 connection(s)
I20260807 15:15:11.607929 22544 log.cc:824] T eb8b9a48cbcc4126aa34d2afe206581a P 246195cca7b043878aaaf04013697524: Log is configured to *not* fsync() on all Append() calls
I20260807 15:15:11.608821 30472 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7fZSIL/build/release/bin/kudu as pid 22462
I20260807 15:15:11.609149 30472 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7fZSIL/build/release/bin/kudu
/tmp/dist-test-task7fZSIL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115480851507-30472-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115480851507-30472-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115480851507-30472-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115480851507-30472-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.194.2:37205
--local_ip_for_outbound_sockets=127.29.194.2
--tserver_master_addrs=127.29.194.62:42465
--webserver_port=36107
--webserver_interface=127.29.194.2
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.2.keytab
--principal=kudu/127.29.194.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.194.20:40443
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115480851507-30472-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115480851507-30472-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115480851507-30472-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 15:15:11.610800 22544 tablet_bootstrap.cc:492] T eb8b9a48cbcc4126aa34d2afe206581a P 246195cca7b043878aaaf04013697524: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 15:15:11.611423 22544 tablet_bootstrap.cc:492] T eb8b9a48cbcc4126aa34d2afe206581a P 246195cca7b043878aaaf04013697524: Bootstrap complete.
I20260807 15:15:11.612576 22544 ts_tablet_manager.cc:1397] T eb8b9a48cbcc4126aa34d2afe206581a P 246195cca7b043878aaaf04013697524: Time spent bootstrapping tablet: real 0.016s user 0.005s sys 0.008s
I20260807 15:15:11.614015 22544 raft_consensus.cc:348] T eb8b9a48cbcc4126aa34d2afe206581a P 246195cca7b043878aaaf04013697524 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d76b0cdebcc44295a6d2a7049ef4b030" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 34775 } } peers { permanent_uuid: "246195cca7b043878aaaf04013697524" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 41283 } } peers { permanent_uuid: "732c0e3bda8644778e6306e1aa2eb6d0" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 37205 } }
I20260807 15:15:11.614333 22544 raft_consensus.cc:729] T eb8b9a48cbcc4126aa34d2afe206581a P 246195cca7b043878aaaf04013697524 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 246195cca7b043878aaaf04013697524, State: Initialized, Role: FOLLOWER
I20260807 15:15:11.614533 22544 consensus_queue.cc:260] T eb8b9a48cbcc4126aa34d2afe206581a P 246195cca7b043878aaaf04013697524 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d76b0cdebcc44295a6d2a7049ef4b030" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 34775 } } peers { permanent_uuid: "246195cca7b043878aaaf04013697524" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 41283 } } peers { permanent_uuid: "732c0e3bda8644778e6306e1aa2eb6d0" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 37205 } }
I20260807 15:15:11.614872 22544 ts_tablet_manager.cc:1428] T eb8b9a48cbcc4126aa34d2afe206581a P 246195cca7b043878aaaf04013697524: Time spent starting tablet: real 0.002s user 0.002s sys 0.003s
W20260807 15:15:11.700256 22703 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:15:11.607682 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:42465 (local address 127.29.194.1:41737)
0807 15:15:11.607867 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:11.607919 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0807 15:15:11.608482 (+ 563us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:15:11.700038 (+ 91556us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.194.62:42465: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":89,"threads_started":1}
W20260807 15:15:11.700538 22699 heartbeater.cc:644] Failed to heartbeat to 127.29.194.62:42465 (0 consecutive failures): Network error: Failed to ping master at 127.29.194.62:42465: Client connection negotiation failed: client connection to 127.29.194.62:42465: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260807 15:15:11.700904 22703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:11.700757 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:42465 (local address 127.29.194.1:53605)
0807 15:15:11.700815 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:11.700870 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.194.62:42465: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260807 15:15:11.701325 22703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:11.701226 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:42465 (local address 127.29.194.1:38645)
0807 15:15:11.701275 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:11.701302 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.194.62:42465: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260807 15:15:11.701506 22699 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260807 15:15:11.706014 22705 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:15:11.706236 22705 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:15:11.706262 22705 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:15:11.707520 22705 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:15:11.707568 22705 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:15:11.707602 22705 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.194.2
I20260807 15:15:11.709182 22705 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.194.20:40443
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115480851507-30472-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115480851507-30472-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.2.keytab
--principal=kudu/127.29.194.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.194.2:37205
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115480851507-30472-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.194.2
--webserver_port=36107
--tserver_master_addrs=127.29.194.62:42465
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22705
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.194.2
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115480851507-30472-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:15:11.709462 22705 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:15:11.709723 22705 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:15:11.715163 22715 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:15:11.715184 22713 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:15:11.715229 22705 server_base.cc:1034] running on GCE node
W20260807 15:15:11.715165 22717 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:15:11.715572 22705 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:15:11.715795 22705 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:15:11.716941 22705 hybrid_clock.cc:648] HybridClock initialized: now 1786115711716924 us; error 34 us; skew 500 ppm
Aug 07 15:15:11 dist-test-slave-42z9 krb5kdc[20839](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115711, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.194.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:15:11.720039 22705 init.cc:377] Logged in from keytab as kudu/127.29.194.2@KRBTEST.COM (short username kudu)
I20260807 15:15:11.720499 22705 webserver.cc:460] Webserver started at http://127.29.194.2:36107/ using document root <none> and password file <none>
I20260807 15:15:11.720736 22705 fs_manager.cc:359] Metadata directory not provided
I20260807 15:15:11.720784 22705 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:15:11.722234 22705 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.000s
I20260807 15:15:11.722879 22727 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:15:11.723040 22705 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260807 15:15:11.723124 22705 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115480851507-30472-0/minicluster-data/ts-1/data,/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115480851507-30472-0/minicluster-data/ts-1/wal
uuid: "732c0e3bda8644778e6306e1aa2eb6d0"
format_stamp: "Formatted at 2026-08-07 15:14:29 on dist-test-slave-42z9"
server_key: "4127a96f8d5a6dfc46b331de9e58b49f"
server_key_iv: "c45eaa31b7403e481f8cbc847079cf50"
server_key_version: "encryptionkey@0"
I20260807 15:15:11.723483 22705 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115480851507-30472-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115480851507-30472-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115480851507-30472-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:15:11.736943 22705 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:15:11.739385 22705 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:15:11.739529 22705 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:15:11.739797 22705 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:15:11.740124 22705 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:15:11.740160 22705 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:15:11.740191 22705 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:15:11.740207 22705 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:15:11.746196 22705 rpc_server.cc:307] RPC server started. Bound to: 127.29.194.2:37205
I20260807 15:15:11.746271 22886 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.194.2:37205 every 8 connection(s)
I20260807 15:15:11.746606 22705 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115480851507-30472-0/minicluster-data/ts-1/data/info.pb
I20260807 15:15:11.747364 30472 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7fZSIL/build/release/bin/kudu as pid 22705
I20260807 15:15:11.747689 30472 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7fZSIL/build/release/bin/kudu
/tmp/dist-test-task7fZSIL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115480851507-30472-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115480851507-30472-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115480851507-30472-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115480851507-30472-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.194.3:34775
--local_ip_for_outbound_sockets=127.29.194.3
--tserver_master_addrs=127.29.194.62:42465
--webserver_port=40145
--webserver_interface=127.29.194.3
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.3.keytab
--principal=kudu/127.29.194.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.194.20:40443
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115480851507-30472-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115480851507-30472-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115480851507-30472-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 15:15:11.747807 22891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:11.747313 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:42465 (local address 127.29.194.2:58949)
0807 15:15:11.747496 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:11.747604 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.194.62:42465: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":71,"threads_started":1}
W20260807 15:15:11.748282 22888 heartbeater.cc:644] Failed to heartbeat to 127.29.194.62:42465 (0 consecutive failures): Network error: Failed to ping master at 127.29.194.62:42465: Client connection negotiation failed: client connection to 127.29.194.62:42465: connect: Connection refused (error 111)
I20260807 15:15:11.748628 22891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:11.748498 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:42465 (local address 127.29.194.2:35513)
0807 15:15:11.748569 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:11.748604 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.194.62:42465: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260807 15:15:11.748909 22891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:11.748836 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:42465 (local address 127.29.194.2:58905)
0807 15:15:11.748872 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:11.748895 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.194.62:42465: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
W20260807 15:15:11.749001 22888 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260807 15:15:11.834645 22892 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:15:11.834860 22892 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:15:11.834885 22892 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:15:11.836035 22892 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:15:11.836073 22892 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:15:11.836099 22892 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.194.3
I20260807 15:15:11.837622 22892 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.194.20:40443
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115480851507-30472-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115480851507-30472-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.3.keytab
--principal=kudu/127.29.194.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.194.3:34775
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115480851507-30472-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.194.3
--webserver_port=40145
--tserver_master_addrs=127.29.194.62:42465
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22892
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.194.3
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115480851507-30472-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:15:11.837823 22892 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:15:11.838017 22892 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 15:15:11.843492 22892 server_base.cc:1034] running on GCE node
W20260807 15:15:11.843438 22898 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:15:11.843438 22905 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:15:11.843374 22903 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:15:11.843920 22892 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:15:11.844164 22892 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:15:11.845340 22892 hybrid_clock.cc:648] HybridClock initialized: now 1786115711845311 us; error 48 us; skew 500 ppm
Aug 07 15:15:11 dist-test-slave-42z9 krb5kdc[20839](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115711, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.194.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:15:11.848065 22892 init.cc:377] Logged in from keytab as kudu/127.29.194.3@KRBTEST.COM (short username kudu)
I20260807 15:15:11.848572 22892 webserver.cc:460] Webserver started at http://127.29.194.3:40145/ using document root <none> and password file <none>
I20260807 15:15:11.848779 22892 fs_manager.cc:359] Metadata directory not provided
I20260807 15:15:11.848830 22892 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:15:11.850289 22892 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260807 15:15:11.851029 22913 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:15:11.851218 22892 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 15:15:11.851310 22892 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115480851507-30472-0/minicluster-data/ts-2/data,/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115480851507-30472-0/minicluster-data/ts-2/wal
uuid: "d76b0cdebcc44295a6d2a7049ef4b030"
format_stamp: "Formatted at 2026-08-07 15:14:30 on dist-test-slave-42z9"
server_key: "e0006eb0c6c241ecdc04fe98b9047bc8"
server_key_iv: "45699283dc90bfe884ea33d509e9b9b3"
server_key_version: "encryptionkey@0"
I20260807 15:15:11.851675 22892 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115480851507-30472-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115480851507-30472-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115480851507-30472-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:15:11.860690 22892 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:15:11.863094 22892 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:15:11.863246 22892 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:15:11.863494 22892 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:15:11.863859 22892 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:15:11.863900 22892 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:15:11.863929 22892 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:15:11.863962 22892 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:15:11.870365 22892 rpc_server.cc:307] RPC server started. Bound to: 127.29.194.3:34775
I20260807 15:15:11.870407 23083 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.194.3:34775 every 8 connection(s)
I20260807 15:15:11.870755 22892 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115480851507-30472-0/minicluster-data/ts-2/data/info.pb
I20260807 15:15:11.871804 23087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:11.871437 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:42465 (local address 127.29.194.3:35107)
0807 15:15:11.871591 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:11.871662 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.194.62:42465: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":80,"threads_started":1}
W20260807 15:15:11.872169 23084 heartbeater.cc:644] Failed to heartbeat to 127.29.194.62:42465 (0 consecutive failures): Network error: Failed to ping master at 127.29.194.62:42465: Client connection negotiation failed: client connection to 127.29.194.62:42465: connect: Connection refused (error 111)
I20260807 15:15:11.872404 23087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:11.872335 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:42465 (local address 127.29.194.3:39405)
0807 15:15:11.872363 (+ 28us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:11.872387 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.194.62:42465: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":15}
I20260807 15:15:11.872601 23087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:11.872553 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:42465 (local address 127.29.194.3:47825)
0807 15:15:11.872574 (+ 21us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:11.872590 (+ 16us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.194.62:42465: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":13}
W20260807 15:15:11.872678 23084 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260807 15:15:11.874991 30472 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7fZSIL/build/release/bin/kudu as pid 22892
I20260807 15:15:12.702198 23090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:12.701908 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:42465 (local address 127.29.194.1:40609)
0807 15:15:12.702077 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:12.702145 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.194.62:42465: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":98,"threads_started":1}
I20260807 15:15:12.749706 23092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:12.749396 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:42465 (local address 127.29.194.2:55457)
0807 15:15:12.749596 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:12.749641 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.194.62:42465: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":133,"threads_started":1}
I20260807 15:15:12.873525 23093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:12.873086 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:42465 (local address 127.29.194.3:58435)
0807 15:15:12.873297 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:12.873371 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.194.62:42465: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":74,"threads_started":1}
I20260807 15:15:13.012373 23096 raft_consensus.cc:482] T eb8b9a48cbcc4126aa34d2afe206581a P 246195cca7b043878aaaf04013697524 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 15:15:13.012615 23096 raft_consensus.cc:504] T eb8b9a48cbcc4126aa34d2afe206581a P 246195cca7b043878aaaf04013697524 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d76b0cdebcc44295a6d2a7049ef4b030" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 34775 } } peers { permanent_uuid: "246195cca7b043878aaaf04013697524" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 41283 } } peers { permanent_uuid: "732c0e3bda8644778e6306e1aa2eb6d0" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 37205 } }
I20260807 15:15:13.012982 23096 leader_election.cc:288] T eb8b9a48cbcc4126aa34d2afe206581a P 246195cca7b043878aaaf04013697524 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d76b0cdebcc44295a6d2a7049ef4b030 (127.29.194.3:34775), 732c0e3bda8644778e6306e1aa2eb6d0 (127.29.194.2:37205)
Aug 07 15:15:13 dist-test-slave-42z9 krb5kdc[20839](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115711, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.194.1@KRBTEST.COM for kudu/127.29.194.3@KRBTEST.COM
Aug 07 15:15:13 dist-test-slave-42z9 krb5kdc[20839](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115711, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.194.1@KRBTEST.COM for kudu/127.29.194.2@KRBTEST.COM
I20260807 15:15:13.025238 23090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:13.013179 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.3:34775 (local address 127.29.194.1:50249)
0807 15:15:13.013287 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:13.013305 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 15:15:13.013386 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:15:13.014283 (+ 897us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:15:13.014287 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:15:13.014739 (+ 452us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:15:13.014938 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:13.014944 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:13.016074 (+ 1130us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:13.016075 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:13.016897 (+ 822us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:13.016906 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:13.017717 (+ 811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:13.017718 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:13.017886 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:13.018575 (+ 689us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:15:13.018587 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:15:13.020399 (+ 1812us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:15:13.022655 (+ 2256us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:15:13.022664 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:15:13.022666 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:15:13.022906 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:15:13.023491 (+ 585us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:15:13.023492 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:15:13.023493 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:15:13.023681 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:15:13.024395 (+ 714us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:15:13.024408 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:15:13.024627 (+ 219us) client_negotiation.cc:769] Sending connection context
0807 15:15:13.024845 (+ 218us) client_negotiation.cc:240] Negotiation successful
0807 15:15:13.025115 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":77,"mutex_wait_us":131}
I20260807 15:15:13.025321 23099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:13.013358 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.194.1:50249 (local address 127.29.194.3:34775)
0807 15:15:13.013521 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:13.013543 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:13.013574 (+ 31us) server_negotiation.cc:407] Connection header received
0807 15:15:13.014037 (+ 463us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:13.014040 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:13.014085 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:13.014208 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:15:13.015097 (+ 889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:13.015946 (+ 849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:13.017058 (+ 1112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:13.017586 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:13.017704 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:13.020544 (+ 2840us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:15:13.020550 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:15:13.020551 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:15:13.020566 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:15:13.022548 (+ 1982us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:15:13.023067 (+ 519us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:15:13.023072 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:15:13.023073 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:15:13.023336 (+ 263us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:15:13.023790 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:15:13.023792 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:15:13.023792 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:15:13.024266 (+ 474us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:15:13.024359 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0807 15:15:13.025007 (+ 648us) server_negotiation.cc:299] Negotiation successful
0807 15:15:13.025190 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":122,"thread_start_us":69,"threads_started":1}
I20260807 15:15:13.025326 23101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:13.013502 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.194.1:52665 (local address 127.29.194.2:37205)
0807 15:15:13.013675 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:13.013683 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:13.013707 (+ 24us) server_negotiation.cc:407] Connection header received
0807 15:15:13.014149 (+ 442us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:13.014151 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:13.014194 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:13.014336 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:15:13.015069 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:13.015945 (+ 876us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:13.016998 (+ 1053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:13.017712 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:13.017872 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:13.020863 (+ 2991us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:15:13.020869 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:15:13.020870 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:15:13.020884 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:15:13.022548 (+ 1664us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:15:13.023232 (+ 684us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:15:13.023236 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:15:13.023236 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:15:13.023398 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:15:13.023808 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:15:13.023809 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:15:13.023809 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:15:13.024239 (+ 430us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:15:13.024359 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0807 15:15:13.025007 (+ 648us) server_negotiation.cc:299] Negotiation successful
0807 15:15:13.025189 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":148,"thread_start_us":88,"threads_started":1}
I20260807 15:15:13.025614 23098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:13.013336 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.2:37205 (local address 127.29.194.1:52665)
0807 15:15:13.013468 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:13.013479 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 15:15:13.013540 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:15:13.014343 (+ 803us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:15:13.014344 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:15:13.014756 (+ 412us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:15:13.014938 (+ 182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:13.014945 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:13.016052 (+ 1107us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:13.016053 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:13.016863 (+ 810us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:13.016865 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:13.017868 (+ 1003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:13.017868 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:13.017917 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:13.018603 (+ 686us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:15:13.018606 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:15:13.020733 (+ 2127us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:15:13.022991 (+ 2258us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:15:13.022992 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:15:13.022992 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:15:13.023119 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:15:13.023496 (+ 377us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:15:13.023497 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:15:13.023497 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:15:13.023681 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:15:13.024388 (+ 707us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:15:13.024408 (+ 20us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:15:13.024627 (+ 219us) client_negotiation.cc:769] Sending connection context
0807 15:15:13.024845 (+ 218us) client_negotiation.cc:240] Negotiation successful
0807 15:15:13.025184 (+ 339us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":97,"mutex_wait_us":174,"thread_start_us":60,"threads_started":1}
W20260807 15:15:13.026757 22533 leader_election.cc:341] T eb8b9a48cbcc4126aa34d2afe206581a P 246195cca7b043878aaaf04013697524 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 732c0e3bda8644778e6306e1aa2eb6d0 (127.29.194.2:37205): Not found: Tablet not found: eb8b9a48cbcc4126aa34d2afe206581a
W20260807 15:15:13.026854 22533 leader_election.cc:341] T eb8b9a48cbcc4126aa34d2afe206581a P 246195cca7b043878aaaf04013697524 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d76b0cdebcc44295a6d2a7049ef4b030 (127.29.194.3:34775): Not found: Tablet not found: eb8b9a48cbcc4126aa34d2afe206581a
I20260807 15:15:13.026877 22533 leader_election.cc:302] T eb8b9a48cbcc4126aa34d2afe206581a P 246195cca7b043878aaaf04013697524 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 246195cca7b043878aaaf04013697524; no voters: 732c0e3bda8644778e6306e1aa2eb6d0, d76b0cdebcc44295a6d2a7049ef4b030
I20260807 15:15:13.027014 23096 raft_consensus.cc:2726] T eb8b9a48cbcc4126aa34d2afe206581a P 246195cca7b043878aaaf04013697524 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 15:15:13.703122 23102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:13.702881 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:42465 (local address 127.29.194.1:36755)
0807 15:15:13.703034 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:13.703074 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.194.62:42465: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":72,"threads_started":1}
I20260807 15:15:13.750396 23103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:13.750180 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:42465 (local address 127.29.194.2:50465)
0807 15:15:13.750311 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:13.750351 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.194.62:42465: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":62,"threads_started":1}
I20260807 15:15:13.874569 23107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:13.874185 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:42465 (local address 127.29.194.3:49245)
0807 15:15:13.874420 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:13.874498 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.194.62:42465: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":70,"threads_started":1}
I20260807 15:15:14.704562 23108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:14.704193 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:42465 (local address 127.29.194.1:60251)
0807 15:15:14.704420 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:14.704487 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.194.62:42465: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":88,"threads_started":1}
I20260807 15:15:14.751067 23110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:14.750826 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:42465 (local address 127.29.194.2:52005)
0807 15:15:14.750980 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:14.751020 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.194.62:42465: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":89,"threads_started":1}
I20260807 15:15:14.875733 23111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:14.875447 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:42465 (local address 127.29.194.3:55139)
0807 15:15:14.875632 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:14.875681 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.194.62:42465: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":82,"threads_started":1}
I20260807 15:15:15.161180 23112 raft_consensus.cc:482] T eb8b9a48cbcc4126aa34d2afe206581a P 246195cca7b043878aaaf04013697524 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 15:15:15.161286 23112 raft_consensus.cc:504] T eb8b9a48cbcc4126aa34d2afe206581a P 246195cca7b043878aaaf04013697524 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d76b0cdebcc44295a6d2a7049ef4b030" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 34775 } } peers { permanent_uuid: "246195cca7b043878aaaf04013697524" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 41283 } } peers { permanent_uuid: "732c0e3bda8644778e6306e1aa2eb6d0" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 37205 } }
I20260807 15:15:15.161496 23112 leader_election.cc:288] T eb8b9a48cbcc4126aa34d2afe206581a P 246195cca7b043878aaaf04013697524 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d76b0cdebcc44295a6d2a7049ef4b030 (127.29.194.3:34775), 732c0e3bda8644778e6306e1aa2eb6d0 (127.29.194.2:37205)
W20260807 15:15:15.162091 22533 leader_election.cc:341] T eb8b9a48cbcc4126aa34d2afe206581a P 246195cca7b043878aaaf04013697524 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d76b0cdebcc44295a6d2a7049ef4b030 (127.29.194.3:34775): Not found: Tablet not found: eb8b9a48cbcc4126aa34d2afe206581a
W20260807 15:15:15.162221 22533 leader_election.cc:341] T eb8b9a48cbcc4126aa34d2afe206581a P 246195cca7b043878aaaf04013697524 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 732c0e3bda8644778e6306e1aa2eb6d0 (127.29.194.2:37205): Not found: Tablet not found: eb8b9a48cbcc4126aa34d2afe206581a
I20260807 15:15:15.162325 22533 leader_election.cc:302] T eb8b9a48cbcc4126aa34d2afe206581a P 246195cca7b043878aaaf04013697524 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 246195cca7b043878aaaf04013697524; no voters: 732c0e3bda8644778e6306e1aa2eb6d0, d76b0cdebcc44295a6d2a7049ef4b030
I20260807 15:15:15.162442 23112 raft_consensus.cc:2726] T eb8b9a48cbcc4126aa34d2afe206581a P 246195cca7b043878aaaf04013697524 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 15:15:15.705703 23113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:15.705292 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:42465 (local address 127.29.194.1:45855)
0807 15:15:15.705543 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:15.705629 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.194.62:42465: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":216,"thread_start_us":73,"threads_started":1}
I20260807 15:15:15.751906 23115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:15.751649 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:42465 (local address 127.29.194.2:45055)
0807 15:15:15.751813 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:15.751858 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.194.62:42465: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":67,"threads_started":1}
I20260807 15:15:15.876497 23116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:15.876260 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:42465 (local address 127.29.194.3:36501)
0807 15:15:15.876411 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:15.876450 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.194.62:42465: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":71,"threads_started":1}
I20260807 15:15:16.706656 23117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:16.706404 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:42465 (local address 127.29.194.1:39117)
0807 15:15:16.706557 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:16.706606 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.194.62:42465: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":73,"threads_started":1}
I20260807 15:15:16.752740 23118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:16.752483 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:42465 (local address 127.29.194.2:39601)
0807 15:15:16.752646 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:16.752693 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.194.62:42465: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":76,"threads_started":1}
I20260807 15:15:16.877400 23121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:16.877108 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:42465 (local address 127.29.194.3:44035)
0807 15:15:16.877304 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:16.877351 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.194.62:42465: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":100,"threads_started":1}
I20260807 15:15:17.707432 23124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:17.707188 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:42465 (local address 127.29.194.1:56853)
0807 15:15:17.707333 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:17.707381 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.194.62:42465: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":69,"threads_started":1}
I20260807 15:15:17.753548 23125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:17.753236 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:42465 (local address 127.29.194.2:38149)
0807 15:15:17.753464 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:17.753501 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.194.62:42465: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":73,"threads_started":1}
I20260807 15:15:17.878266 23127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:17.877997 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:42465 (local address 127.29.194.3:33887)
0807 15:15:17.878157 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:17.878206 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.194.62:42465: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":69,"threads_started":1}
I20260807 15:15:17.907040 23128 raft_consensus.cc:482] T eb8b9a48cbcc4126aa34d2afe206581a P 246195cca7b043878aaaf04013697524 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 15:15:17.907238 23128 raft_consensus.cc:504] T eb8b9a48cbcc4126aa34d2afe206581a P 246195cca7b043878aaaf04013697524 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d76b0cdebcc44295a6d2a7049ef4b030" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 34775 } } peers { permanent_uuid: "246195cca7b043878aaaf04013697524" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 41283 } } peers { permanent_uuid: "732c0e3bda8644778e6306e1aa2eb6d0" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 37205 } }
I20260807 15:15:17.907491 23128 leader_election.cc:288] T eb8b9a48cbcc4126aa34d2afe206581a P 246195cca7b043878aaaf04013697524 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d76b0cdebcc44295a6d2a7049ef4b030 (127.29.194.3:34775), 732c0e3bda8644778e6306e1aa2eb6d0 (127.29.194.2:37205)
W20260807 15:15:17.907984 22533 leader_election.cc:341] T eb8b9a48cbcc4126aa34d2afe206581a P 246195cca7b043878aaaf04013697524 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d76b0cdebcc44295a6d2a7049ef4b030 (127.29.194.3:34775): Not found: Tablet not found: eb8b9a48cbcc4126aa34d2afe206581a
W20260807 15:15:17.908098 22533 leader_election.cc:341] T eb8b9a48cbcc4126aa34d2afe206581a P 246195cca7b043878aaaf04013697524 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 732c0e3bda8644778e6306e1aa2eb6d0 (127.29.194.2:37205): Not found: Tablet not found: eb8b9a48cbcc4126aa34d2afe206581a
I20260807 15:15:17.908133 22533 leader_election.cc:302] T eb8b9a48cbcc4126aa34d2afe206581a P 246195cca7b043878aaaf04013697524 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 246195cca7b043878aaaf04013697524; no voters: 732c0e3bda8644778e6306e1aa2eb6d0, d76b0cdebcc44295a6d2a7049ef4b030
I20260807 15:15:17.908245 23128 raft_consensus.cc:2726] T eb8b9a48cbcc4126aa34d2afe206581a P 246195cca7b043878aaaf04013697524 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 15:15:18.708343 23130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:18.708061 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:42465 (local address 127.29.194.1:60601)
0807 15:15:18.708240 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:18.708291 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.194.62:42465: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":95,"threads_started":1}
I20260807 15:15:18.754379 23131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:18.754068 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:42465 (local address 127.29.194.2:53589)
0807 15:15:18.754258 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:18.754310 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.194.62:42465: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":66,"threads_started":1}
I20260807 15:15:18.879113 23132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:18.878871 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:42465 (local address 127.29.194.3:42225)
0807 15:15:18.879027 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:18.879064 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.194.62:42465: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":79,"threads_started":1}
I20260807 15:15:19.709210 23133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:19.708974 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:42465 (local address 127.29.194.1:54737)
0807 15:15:19.709112 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:19.709154 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.194.62:42465: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":75,"threads_started":1}
I20260807 15:15:19.755167 23134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:19.754909 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:42465 (local address 127.29.194.2:43775)
0807 15:15:19.755068 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:19.755113 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.194.62:42465: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":86,"threads_started":1}
I20260807 15:15:19.880105 23138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:19.879795 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:42465 (local address 127.29.194.3:47531)
0807 15:15:19.879926 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:19.879964 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.194.62:42465: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":66,"threads_started":1}
I20260807 15:15:20.710004 23139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:20.709754 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:42465 (local address 127.29.194.1:43503)
0807 15:15:20.709907 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:20.709956 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.194.62:42465: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":70,"threads_started":1}
I20260807 15:15:20.755987 23140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:20.755752 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:42465 (local address 127.29.194.2:60099)
0807 15:15:20.755881 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:20.755925 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.194.62:42465: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":68,"threads_started":1}
I20260807 15:15:20.880901 23141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:20.880638 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:42465 (local address 127.29.194.3:48175)
0807 15:15:20.880796 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:20.880847 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.194.62:42465: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":66,"threads_started":1}
I20260807 15:15:21.710924 23142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:21.710657 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:42465 (local address 127.29.194.1:54963)
0807 15:15:21.710825 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:21.710874 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.194.62:42465: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":75,"threads_started":1}
I20260807 15:15:21.756767 23143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:21.756515 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:42465 (local address 127.29.194.2:55071)
0807 15:15:21.756660 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:21.756719 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.194.62:42465: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":71,"threads_started":1}
I20260807 15:15:21.881747 23145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:21.881498 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:42465 (local address 127.29.194.3:51809)
0807 15:15:21.881649 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:21.881696 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.194.62:42465: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":84,"threads_started":1}
I20260807 15:15:22.344072 23146 raft_consensus.cc:482] T eb8b9a48cbcc4126aa34d2afe206581a P 246195cca7b043878aaaf04013697524 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 15:15:22.344237 23146 raft_consensus.cc:504] T eb8b9a48cbcc4126aa34d2afe206581a P 246195cca7b043878aaaf04013697524 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d76b0cdebcc44295a6d2a7049ef4b030" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 34775 } } peers { permanent_uuid: "246195cca7b043878aaaf04013697524" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 41283 } } peers { permanent_uuid: "732c0e3bda8644778e6306e1aa2eb6d0" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 37205 } }
I20260807 15:15:22.344466 23146 leader_election.cc:288] T eb8b9a48cbcc4126aa34d2afe206581a P 246195cca7b043878aaaf04013697524 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d76b0cdebcc44295a6d2a7049ef4b030 (127.29.194.3:34775), 732c0e3bda8644778e6306e1aa2eb6d0 (127.29.194.2:37205)
W20260807 15:15:22.344964 22533 leader_election.cc:341] T eb8b9a48cbcc4126aa34d2afe206581a P 246195cca7b043878aaaf04013697524 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d76b0cdebcc44295a6d2a7049ef4b030 (127.29.194.3:34775): Not found: Tablet not found: eb8b9a48cbcc4126aa34d2afe206581a
W20260807 15:15:22.345077 22533 leader_election.cc:341] T eb8b9a48cbcc4126aa34d2afe206581a P 246195cca7b043878aaaf04013697524 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 732c0e3bda8644778e6306e1aa2eb6d0 (127.29.194.2:37205): Not found: Tablet not found: eb8b9a48cbcc4126aa34d2afe206581a
I20260807 15:15:22.345096 22533 leader_election.cc:302] T eb8b9a48cbcc4126aa34d2afe206581a P 246195cca7b043878aaaf04013697524 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 246195cca7b043878aaaf04013697524; no voters: 732c0e3bda8644778e6306e1aa2eb6d0, d76b0cdebcc44295a6d2a7049ef4b030
I20260807 15:15:22.345180 23146 raft_consensus.cc:2726] T eb8b9a48cbcc4126aa34d2afe206581a P 246195cca7b043878aaaf04013697524 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 15:15:22.711796 23147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:22.711529 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:42465 (local address 127.29.194.1:46703)
0807 15:15:22.711694 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:22.711745 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.194.62:42465: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":73,"threads_started":1}
I20260807 15:15:22.757707 23149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:22.757352 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:42465 (local address 127.29.194.2:48011)
0807 15:15:22.757604 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:22.757653 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.194.62:42465: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":230,"thread_start_us":207,"threads_started":1}
I20260807 15:15:22.882946 23150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:22.882510 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:42465 (local address 127.29.194.3:37523)
0807 15:15:22.882816 (+ 306us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:22.882872 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.194.62:42465: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":276,"thread_start_us":197,"threads_started":1}
I20260807 15:15:23.712910 23151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:23.712542 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:42465 (local address 127.29.194.1:54301)
0807 15:15:23.712799 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:23.712857 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.194.62:42465: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":228,"thread_start_us":135,"threads_started":1}
I20260807 15:15:23.758744 23152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:23.758389 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:42465 (local address 127.29.194.2:42749)
0807 15:15:23.758618 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:23.758677 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.194.62:42465: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":105,"threads_started":1}
I20260807 15:15:23.884003 23155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:23.883728 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:42465 (local address 127.29.194.3:41203)
0807 15:15:23.883885 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:23.883943 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.194.62:42465: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":77,"threads_started":1}
I20260807 15:15:24.713701 23156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:24.713494 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:42465 (local address 127.29.194.1:58723)
0807 15:15:24.713617 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:24.713652 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.194.62:42465: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":64,"threads_started":1}
I20260807 15:15:24.759738 23157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:24.759415 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:42465 (local address 127.29.194.2:46735)
0807 15:15:24.759644 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:24.759689 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.194.62:42465: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":95,"threads_started":1}
I20260807 15:15:24.884840 23158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:24.884621 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:42465 (local address 127.29.194.3:51949)
0807 15:15:24.884755 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:24.884792 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.194.62:42465: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":83,"threads_started":1}
I20260807 15:15:25.135170 23159 raft_consensus.cc:482] T eb8b9a48cbcc4126aa34d2afe206581a P 246195cca7b043878aaaf04013697524 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 15:15:25.135281 23159 raft_consensus.cc:504] T eb8b9a48cbcc4126aa34d2afe206581a P 246195cca7b043878aaaf04013697524 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d76b0cdebcc44295a6d2a7049ef4b030" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 34775 } } peers { permanent_uuid: "246195cca7b043878aaaf04013697524" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 41283 } } peers { permanent_uuid: "732c0e3bda8644778e6306e1aa2eb6d0" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 37205 } }
I20260807 15:15:25.135456 23159 leader_election.cc:288] T eb8b9a48cbcc4126aa34d2afe206581a P 246195cca7b043878aaaf04013697524 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d76b0cdebcc44295a6d2a7049ef4b030 (127.29.194.3:34775), 732c0e3bda8644778e6306e1aa2eb6d0 (127.29.194.2:37205)
W20260807 15:15:25.135890 22533 leader_election.cc:341] T eb8b9a48cbcc4126aa34d2afe206581a P 246195cca7b043878aaaf04013697524 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d76b0cdebcc44295a6d2a7049ef4b030 (127.29.194.3:34775): Not found: Tablet not found: eb8b9a48cbcc4126aa34d2afe206581a
W20260807 15:15:25.135967 22533 leader_election.cc:341] T eb8b9a48cbcc4126aa34d2afe206581a P 246195cca7b043878aaaf04013697524 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 732c0e3bda8644778e6306e1aa2eb6d0 (127.29.194.2:37205): Not found: Tablet not found: eb8b9a48cbcc4126aa34d2afe206581a
I20260807 15:15:25.135995 22533 leader_election.cc:302] T eb8b9a48cbcc4126aa34d2afe206581a P 246195cca7b043878aaaf04013697524 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 246195cca7b043878aaaf04013697524; no voters: 732c0e3bda8644778e6306e1aa2eb6d0, d76b0cdebcc44295a6d2a7049ef4b030
I20260807 15:15:25.136054 23159 raft_consensus.cc:2726] T eb8b9a48cbcc4126aa34d2afe206581a P 246195cca7b043878aaaf04013697524 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 15:15:25.714738 23162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:25.714379 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:42465 (local address 127.29.194.1:43509)
0807 15:15:25.714619 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:25.714684 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.194.62:42465: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":94,"threads_started":1}
I20260807 15:15:25.760694 23164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:25.760390 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:42465 (local address 127.29.194.2:60173)
0807 15:15:25.760555 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:25.760589 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.194.62:42465: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":102,"threads_started":1}
I20260807 15:15:25.885634 23166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:25.885363 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:42465 (local address 127.29.194.3:42853)
0807 15:15:25.885542 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:25.885584 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.194.62:42465: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":109,"threads_started":1}
I20260807 15:15:26.715687 23169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:26.715453 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:42465 (local address 127.29.194.1:45415)
0807 15:15:26.715596 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:26.715638 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.194.62:42465: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":69,"threads_started":1}
I20260807 15:15:26.761924 23170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:26.761586 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:42465 (local address 127.29.194.2:37865)
0807 15:15:26.761821 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:26.761871 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.194.62:42465: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":215,"thread_start_us":147,"threads_started":1}
I20260807 15:15:26.877002 30472 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7fZSIL/build/release/bin/kudu with pid 22462
I20260807 15:15:26.886479 23171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:26.886205 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:42465 (local address 127.29.194.3:38633)
0807 15:15:26.886371 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:26.886430 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.194.62:42465: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":68,"threads_started":1}
I20260807 15:15:26.888156 30472 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7fZSIL/build/release/bin/kudu with pid 22705
I20260807 15:15:26.899325 30472 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7fZSIL/build/release/bin/kudu with pid 22892
2026-08-07T15:15:26Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (58723 ms)
[----------] 30 tests from SecurityITest (246059 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115480851507-30472-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 15:15:26 dist-test-slave-42z9 krb5kdc[23178](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 15:15:26 dist-test-slave-42z9 krb5kdc[23178](info): set up 2 sockets
Aug 07 15:15:26 dist-test-slave-42z9 krb5kdc[23178](info): commencing operation
krb5kdc: starting...
W20260807 15:15:28.059403 30472 mini_kdc.cc:121] Time spent starting KDC: real 1.133s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115480851507-30472-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115480851507-30472-0/krb5kdc/test-user.keytab.
Aug 07 15:15:28 dist-test-slave-42z9 krb5kdc[23178](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115728, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T15:15:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T15:15:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.194.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.194.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.194.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.62.keytab.
Entry for principal kudu/127.29.194.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.62.keytab.
I20260807 15:15:28.193166 30472 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7fZSIL/build/release/bin/kudu
/tmp/dist-test-task7fZSIL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115480851507-30472-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115480851507-30472-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115480851507-30472-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115480851507-30472-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.194.62:37251
--webserver_interface=127.29.194.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.62.keytab
--principal=kudu/127.29.194.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.194.20:46859
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.194.62:37251
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115480851507-30472-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115480851507-30472-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115480851507-30472-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:15:28.279420 23206 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:15:28.279655 23206 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:15:28.279680 23206 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:15:28.281983 23206 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:15:28.282097 23206 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:15:28.282114 23206 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:15:28.282126 23206 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:15:28.282141 23206 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:15:28.285154 23206 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.194.20:46859
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115480851507-30472-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115480851507-30472-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.194.62:37251
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.62.keytab
--principal=kudu/127.29.194.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.194.62:37251
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115480851507-30472-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.194.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23206
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115480851507-30472-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:15:28.285526 23206 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:15:28.285813 23206 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:15:28.292142 23214 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:15:28.292224 23206 server_base.cc:1034] running on GCE node
W20260807 15:15:28.292140 23217 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:15:28.292133 23213 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:15:28.292657 23206 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:15:28.292969 23206 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:15:28.294116 23206 hybrid_clock.cc:648] HybridClock initialized: now 1786115728294104 us; error 32 us; skew 500 ppm
Aug 07 15:15:28 dist-test-slave-42z9 krb5kdc[23178](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115728, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.194.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:15:28.297175 23206 init.cc:377] Logged in from keytab as kudu/127.29.194.62@KRBTEST.COM (short username kudu)
I20260807 15:15:28.297716 23206 webserver.cc:460] Webserver started at http://127.29.194.62:41613/ using document root <none> and password file <none>
I20260807 15:15:28.297948 23206 fs_manager.cc:359] Metadata directory not provided
I20260807 15:15:28.297997 23206 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:15:28.298128 23206 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:15:28.299206 23206 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115480851507-30472-0/minicluster-data/master-0/data/instance:
uuid: "832fb5572e1b48ab9521c052b74260c2"
format_stamp: "Formatted at 2026-08-07 15:15:28 on dist-test-slave-42z9"
server_key: "663ab9c6a8000f9f4f1c2b4f78b2f5b6"
server_key_iv: "72effaf4f96f8733cca07f6e34783902"
server_key_version: "encryptionkey@0"
I20260807 15:15:28.299573 23206 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115480851507-30472-0/minicluster-data/master-0/wal/instance:
uuid: "832fb5572e1b48ab9521c052b74260c2"
format_stamp: "Formatted at 2026-08-07 15:15:28 on dist-test-slave-42z9"
server_key: "663ab9c6a8000f9f4f1c2b4f78b2f5b6"
server_key_iv: "72effaf4f96f8733cca07f6e34783902"
server_key_version: "encryptionkey@0"
I20260807 15:15:28.301095 23206 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 15:15:28.302425 23228 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:15:28.302740 23206 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 15:15:28.302858 23206 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115480851507-30472-0/minicluster-data/master-0/data,/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115480851507-30472-0/minicluster-data/master-0/wal
uuid: "832fb5572e1b48ab9521c052b74260c2"
format_stamp: "Formatted at 2026-08-07 15:15:28 on dist-test-slave-42z9"
server_key: "663ab9c6a8000f9f4f1c2b4f78b2f5b6"
server_key_iv: "72effaf4f96f8733cca07f6e34783902"
server_key_version: "encryptionkey@0"
I20260807 15:15:28.302935 23206 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115480851507-30472-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115480851507-30472-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115480851507-30472-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:15:28.320853 23206 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:15:28.323722 23206 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:15:28.323863 23206 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:15:28.329010 23206 rpc_server.cc:307] RPC server started. Bound to: 127.29.194.62:37251
I20260807 15:15:28.329066 23312 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.194.62:37251 every 8 connection(s)
I20260807 15:15:28.329561 23206 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115480851507-30472-0/minicluster-data/master-0/data/info.pb
I20260807 15:15:28.330097 30472 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7fZSIL/build/release/bin/kudu as pid 23206
I20260807 15:15:28.330200 30472 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115480851507-30472-0/minicluster-data/master-0/wal/instance
I20260807 15:15:28.330417 30472 external_mini_cluster.cc:1428] Setting key 4c1093ec822a25b5653601655298df9c
I20260807 15:15:28.330724 23314 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:15:28.334388 23314 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 832fb5572e1b48ab9521c052b74260c2: Bootstrap starting.
I20260807 15:15:28.335069 23314 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 832fb5572e1b48ab9521c052b74260c2: Neither blocks nor log segments found. Creating new log.
I20260807 15:15:28.335417 23314 log.cc:824] T 00000000000000000000000000000000 P 832fb5572e1b48ab9521c052b74260c2: Log is configured to *not* fsync() on all Append() calls
I20260807 15:15:28.336231 23314 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 832fb5572e1b48ab9521c052b74260c2: No bootstrap required, opened a new log
Aug 07 15:15:28 dist-test-slave-42z9 krb5kdc[23178](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115728, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.194.62@KRBTEST.COM
I20260807 15:15:28.338711 23314 raft_consensus.cc:348] T 00000000000000000000000000000000 P 832fb5572e1b48ab9521c052b74260c2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "832fb5572e1b48ab9521c052b74260c2" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 37251 } }
I20260807 15:15:28.338902 23314 raft_consensus.cc:374] T 00000000000000000000000000000000 P 832fb5572e1b48ab9521c052b74260c2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:15:28.338955 23314 raft_consensus.cc:729] T 00000000000000000000000000000000 P 832fb5572e1b48ab9521c052b74260c2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 832fb5572e1b48ab9521c052b74260c2, State: Initialized, Role: FOLLOWER
I20260807 15:15:28.339097 23314 consensus_queue.cc:260] T 00000000000000000000000000000000 P 832fb5572e1b48ab9521c052b74260c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "832fb5572e1b48ab9521c052b74260c2" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 37251 } }
I20260807 15:15:28.339178 23314 raft_consensus.cc:388] T 00000000000000000000000000000000 P 832fb5572e1b48ab9521c052b74260c2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:15:28.339222 23314 raft_consensus.cc:482] T 00000000000000000000000000000000 P 832fb5572e1b48ab9521c052b74260c2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:15:28.339264 23314 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 832fb5572e1b48ab9521c052b74260c2 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:15:28.339906 23314 raft_consensus.cc:504] T 00000000000000000000000000000000 P 832fb5572e1b48ab9521c052b74260c2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "832fb5572e1b48ab9521c052b74260c2" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 37251 } }
I20260807 15:15:28.339999 23314 leader_election.cc:302] T 00000000000000000000000000000000 P 832fb5572e1b48ab9521c052b74260c2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 832fb5572e1b48ab9521c052b74260c2; no voters:
I20260807 15:15:28.340179 23314 leader_election.cc:288] T 00000000000000000000000000000000 P 832fb5572e1b48ab9521c052b74260c2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 15:15:28.340231 23320 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 832fb5572e1b48ab9521c052b74260c2 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:15:28.340395 23320 raft_consensus.cc:686] T 00000000000000000000000000000000 P 832fb5572e1b48ab9521c052b74260c2 [term 1 LEADER]: Becoming Leader. State: Replica: 832fb5572e1b48ab9521c052b74260c2, State: Running, Role: LEADER
I20260807 15:15:28.340509 23314 sys_catalog.cc:564] T 00000000000000000000000000000000 P 832fb5572e1b48ab9521c052b74260c2 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:15:28.340534 23320 consensus_queue.cc:237] T 00000000000000000000000000000000 P 832fb5572e1b48ab9521c052b74260c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "832fb5572e1b48ab9521c052b74260c2" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 37251 } }
I20260807 15:15:28.340932 23320 sys_catalog.cc:455] T 00000000000000000000000000000000 P 832fb5572e1b48ab9521c052b74260c2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 832fb5572e1b48ab9521c052b74260c2. Latest consensus state: current_term: 1 leader_uuid: "832fb5572e1b48ab9521c052b74260c2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "832fb5572e1b48ab9521c052b74260c2" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 37251 } } }
I20260807 15:15:28.341104 23320 sys_catalog.cc:458] T 00000000000000000000000000000000 P 832fb5572e1b48ab9521c052b74260c2 [sys.catalog]: This master's current role is: LEADER
I20260807 15:15:28.340966 23325 sys_catalog.cc:455] T 00000000000000000000000000000000 P 832fb5572e1b48ab9521c052b74260c2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "832fb5572e1b48ab9521c052b74260c2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "832fb5572e1b48ab9521c052b74260c2" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 37251 } } }
I20260807 15:15:28.341348 23325 sys_catalog.cc:458] T 00000000000000000000000000000000 P 832fb5572e1b48ab9521c052b74260c2 [sys.catalog]: This master's current role is: LEADER
I20260807 15:15:28.341749 23334 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:15:28.342320 23334 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:15:28.342779 23318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:28.331096 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33724 (local address 127.29.194.62:37251)
0807 15:15:28.331308 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:28.331317 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:28.331351 (+ 34us) server_negotiation.cc:407] Connection header received
0807 15:15:28.331981 (+ 630us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:28.331991 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:28.332060 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:28.332221 (+ 161us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:15:28.332872 (+ 651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:28.334258 (+ 1386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:28.335104 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:28.335701 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:28.335797 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:28.338571 (+ 2774us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:15:28.338577 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:15:28.338577 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:15:28.338593 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:15:28.340486 (+ 1893us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:15:28.341080 (+ 594us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:15:28.341083 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:15:28.341084 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:15:28.341195 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:15:28.341468 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:15:28.341469 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:15:28.341469 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:15:28.341819 (+ 350us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:15:28.342402 (+ 583us) server_negotiation.cc:1035] Waiting for connection context
0807 15:15:28.342480 (+ 78us) server_negotiation.cc:299] Negotiation successful
0807 15:15:28.342609 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":152,"thread_start_us":94,"threads_started":1}
I20260807 15:15:28.344568 23334 catalog_manager.cc:1347] Generated new cluster ID: 57e717a0c31e48b080a7e002b505d161
I20260807 15:15:28.344637 23334 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:15:28.353622 23334 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:15:28.354195 23334 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:15:28.365238 23334 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 832fb5572e1b48ab9521c052b74260c2: Generated new TSK 0
I20260807 15:15:28.365477 23334 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.29.194.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.194.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.194.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.1.keytab.
Entry for principal kudu/127.29.194.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.1.keytab.
I20260807 15:15:28.400695 30472 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7fZSIL/build/release/bin/kudu
/tmp/dist-test-task7fZSIL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115480851507-30472-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115480851507-30472-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115480851507-30472-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115480851507-30472-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.194.1:0
--local_ip_for_outbound_sockets=127.29.194.1
--webserver_interface=127.29.194.1
--webserver_port=0
--tserver_master_addrs=127.29.194.62:37251
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.1.keytab
--principal=kudu/127.29.194.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.194.20:46859
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115480851507-30472-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115480851507-30472-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115480851507-30472-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:15:28.519443 23352 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:15:28.519671 23352 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:15:28.519706 23352 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:15:28.522123 23352 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:15:28.522198 23352 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:15:28.522248 23352 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.194.1
I20260807 15:15:28.524480 23352 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.194.20:46859
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115480851507-30472-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115480851507-30472-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.1.keytab
--principal=kudu/127.29.194.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.194.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115480851507-30472-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.194.1
--webserver_port=0
--tserver_master_addrs=127.29.194.62:37251
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23352
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.194.1
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115480851507-30472-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:15:28.524770 23352 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:15:28.525022 23352 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:15:28.530385 23359 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:15:28.530388 23361 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:15:28.530385 23364 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:15:28.530741 23352 server_base.cc:1034] running on GCE node
I20260807 15:15:28.530987 23352 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:15:28.531298 23352 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:15:28.532467 23352 hybrid_clock.cc:648] HybridClock initialized: now 1786115728532447 us; error 37 us; skew 500 ppm
Aug 07 15:15:28 dist-test-slave-42z9 krb5kdc[23178](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115728, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.194.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:15:28.535862 23352 init.cc:377] Logged in from keytab as kudu/127.29.194.1@KRBTEST.COM (short username kudu)
I20260807 15:15:28.536494 23352 webserver.cc:460] Webserver started at http://127.29.194.1:38101/ using document root <none> and password file <none>
I20260807 15:15:28.536785 23352 fs_manager.cc:359] Metadata directory not provided
I20260807 15:15:28.536851 23352 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:15:28.536995 23352 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:15:28.538494 23352 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115480851507-30472-0/minicluster-data/ts-0/data/instance:
uuid: "2a50a320ccb04b85bbb55d30db39721f"
format_stamp: "Formatted at 2026-08-07 15:15:28 on dist-test-slave-42z9"
server_key: "beeda639b2b42b51b373b662bafb7a74"
server_key_iv: "e601e8aa5920c2d198719a83462c5c0a"
server_key_version: "encryptionkey@0"
I20260807 15:15:28.538918 23352 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115480851507-30472-0/minicluster-data/ts-0/wal/instance:
uuid: "2a50a320ccb04b85bbb55d30db39721f"
format_stamp: "Formatted at 2026-08-07 15:15:28 on dist-test-slave-42z9"
server_key: "beeda639b2b42b51b373b662bafb7a74"
server_key_iv: "e601e8aa5920c2d198719a83462c5c0a"
server_key_version: "encryptionkey@0"
I20260807 15:15:28.540953 23352 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.003s
I20260807 15:15:28.542057 23373 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:15:28.542258 23352 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260807 15:15:28.542379 23352 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115480851507-30472-0/minicluster-data/ts-0/data,/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115480851507-30472-0/minicluster-data/ts-0/wal
uuid: "2a50a320ccb04b85bbb55d30db39721f"
format_stamp: "Formatted at 2026-08-07 15:15:28 on dist-test-slave-42z9"
server_key: "beeda639b2b42b51b373b662bafb7a74"
server_key_iv: "e601e8aa5920c2d198719a83462c5c0a"
server_key_version: "encryptionkey@0"
I20260807 15:15:28.542464 23352 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115480851507-30472-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115480851507-30472-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115480851507-30472-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:15:28.551959 23352 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:15:28.554430 23352 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:15:28.554593 23352 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:15:28.554800 23352 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:15:28.555101 23352 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:15:28.555137 23352 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:15:28.555166 23352 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:15:28.555190 23352 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:15:28.561923 23352 rpc_server.cc:307] RPC server started. Bound to: 127.29.194.1:42461
I20260807 15:15:28.562426 23352 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115480851507-30472-0/minicluster-data/ts-0/data/info.pb
I20260807 15:15:28.562929 30472 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7fZSIL/build/release/bin/kudu as pid 23352
I20260807 15:15:28.563040 30472 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115480851507-30472-0/minicluster-data/ts-0/wal/instance
I20260807 15:15:28.563252 30472 external_mini_cluster.cc:1428] Setting key 94c78c13989e017b99599c4890d1505e
I20260807 15:15:28.567947 23546 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.194.1:42461 every 8 connection(s)
Aug 07 15:15:28 dist-test-slave-42z9 krb5kdc[23178](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115728, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.194.1@KRBTEST.COM for kudu/127.29.194.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.29.194.2@KRBTEST.COM; defaulting to no policy
I20260807 15:15:28.579020 23318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:28.568725 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.194.1:44203 (local address 127.29.194.62:37251)
0807 15:15:28.568770 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:28.568772 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:28.569058 (+ 286us) server_negotiation.cc:407] Connection header received
0807 15:15:28.569798 (+ 740us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:28.569800 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:28.569812 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:28.569840 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:15:28.571194 (+ 1354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:28.571884 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:28.573481 (+ 1597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:28.573799 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:28.573881 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:28.576487 (+ 2606us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:15:28.576489 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:15:28.576489 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:15:28.576497 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:15:28.577741 (+ 1244us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:15:28.578104 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:15:28.578105 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:15:28.578105 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:15:28.578150 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:15:28.578371 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:15:28.578372 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:15:28.578372 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:15:28.578499 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:15:28.578565 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0807 15:15:28.578856 (+ 291us) server_negotiation.cc:299] Negotiation successful
0807 15:15:28.578945 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260807 15:15:28.579185 23552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:28.568717 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:37251 (local address 127.29.194.1:44203)
0807 15:15:28.568921 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:28.568977 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0807 15:15:28.569679 (+ 702us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:15:28.570012 (+ 333us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:15:28.570017 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:15:28.570788 (+ 771us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:15:28.571061 (+ 273us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:28.571067 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:28.572007 (+ 940us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:28.572009 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:28.573344 (+ 1335us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:28.573348 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:28.573920 (+ 572us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:28.573922 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:28.574100 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:28.574755 (+ 655us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:15:28.574765 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:15:28.576376 (+ 1611us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:15:28.577807 (+ 1431us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:15:28.577815 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:15:28.577817 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:15:28.578043 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:15:28.578194 (+ 151us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:15:28.578194 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:15:28.578194 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:15:28.578290 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:15:28.578558 (+ 268us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:15:28.578562 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:15:28.578719 (+ 157us) client_negotiation.cc:769] Sending connection context
0807 15:15:28.578834 (+ 115us) client_negotiation.cc:240] Negotiation successful
0807 15:15:28.578968 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":112,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.194.2@KRBTEST.COM" created.
I20260807 15:15:28.579846 23549 heartbeater.cc:347] Connected to a master server at 127.29.194.62:37251
I20260807 15:15:28.579970 23549 heartbeater.cc:464] Registering TS with master...
I20260807 15:15:28.580168 23549 heartbeater.cc:511] Master 127.29.194.62:37251 requested a full tablet report, sending...
I20260807 15:15:28.580713 23257 ts_manager.cc:194] Registered new tserver with Master: 2a50a320ccb04b85bbb55d30db39721f (127.29.194.1:42461)
I20260807 15:15:28.581310 23257 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.194.1@KRBTEST.COM'} at 127.29.194.1:44203
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.194.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.2.keytab.
Entry for principal kudu/127.29.194.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.2.keytab.
I20260807 15:15:28.596711 30472 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7fZSIL/build/release/bin/kudu
/tmp/dist-test-task7fZSIL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115480851507-30472-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115480851507-30472-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115480851507-30472-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115480851507-30472-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.194.2:0
--local_ip_for_outbound_sockets=127.29.194.2
--webserver_interface=127.29.194.2
--webserver_port=0
--tserver_master_addrs=127.29.194.62:37251
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.2.keytab
--principal=kudu/127.29.194.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.194.20:46859
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115480851507-30472-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115480851507-30472-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115480851507-30472-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:15:28.681054 23557 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:15:28.681277 23557 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:15:28.681308 23557 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:15:28.682562 23557 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:15:28.682610 23557 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:15:28.682636 23557 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.194.2
I20260807 15:15:28.684173 23557 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.194.20:46859
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115480851507-30472-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115480851507-30472-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.2.keytab
--principal=kudu/127.29.194.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.194.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115480851507-30472-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.194.2
--webserver_port=0
--tserver_master_addrs=127.29.194.62:37251
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23557
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.194.2
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115480851507-30472-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:15:28.684394 23557 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:15:28.684641 23557 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:15:28.689664 23563 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:15:28.689685 23568 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:15:28.689658 23564 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:15:28.689670 23557 server_base.cc:1034] running on GCE node
I20260807 15:15:28.690099 23557 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:15:28.690306 23557 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:15:28.691435 23557 hybrid_clock.cc:648] HybridClock initialized: now 1786115728691424 us; error 27 us; skew 500 ppm
Aug 07 15:15:28 dist-test-slave-42z9 krb5kdc[23178](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115728, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.194.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:15:28.693905 23557 init.cc:377] Logged in from keytab as kudu/127.29.194.2@KRBTEST.COM (short username kudu)
I20260807 15:15:28.694397 23557 webserver.cc:460] Webserver started at http://127.29.194.2:35089/ using document root <none> and password file <none>
I20260807 15:15:28.694618 23557 fs_manager.cc:359] Metadata directory not provided
I20260807 15:15:28.694658 23557 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:15:28.694756 23557 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:15:28.695761 23557 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115480851507-30472-0/minicluster-data/ts-1/data/instance:
uuid: "3a06c83f62054947a37122c46d1206be"
format_stamp: "Formatted at 2026-08-07 15:15:28 on dist-test-slave-42z9"
server_key: "ea8376b1a8d931a6a956d644637e7862"
server_key_iv: "1a8c422cab7c45f2d3309dd6ff3695d6"
server_key_version: "encryptionkey@0"
I20260807 15:15:28.696053 23557 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115480851507-30472-0/minicluster-data/ts-1/wal/instance:
uuid: "3a06c83f62054947a37122c46d1206be"
format_stamp: "Formatted at 2026-08-07 15:15:28 on dist-test-slave-42z9"
server_key: "ea8376b1a8d931a6a956d644637e7862"
server_key_iv: "1a8c422cab7c45f2d3309dd6ff3695d6"
server_key_version: "encryptionkey@0"
I20260807 15:15:28.697252 23557 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260807 15:15:28.697960 23575 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:15:28.698101 23557 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260807 15:15:28.698184 23557 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115480851507-30472-0/minicluster-data/ts-1/data,/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115480851507-30472-0/minicluster-data/ts-1/wal
uuid: "3a06c83f62054947a37122c46d1206be"
format_stamp: "Formatted at 2026-08-07 15:15:28 on dist-test-slave-42z9"
server_key: "ea8376b1a8d931a6a956d644637e7862"
server_key_iv: "1a8c422cab7c45f2d3309dd6ff3695d6"
server_key_version: "encryptionkey@0"
I20260807 15:15:28.698246 23557 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115480851507-30472-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115480851507-30472-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115480851507-30472-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:15:28.723534 23557 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:15:28.725806 23557 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:15:28.725970 23557 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:15:28.726227 23557 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:15:28.726683 23557 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:15:28.726738 23557 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:15:28.726773 23557 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:15:28.726791 23557 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:15:28.732540 23557 rpc_server.cc:307] RPC server started. Bound to: 127.29.194.2:33907
I20260807 15:15:28.732596 23743 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.194.2:33907 every 8 connection(s)
I20260807 15:15:28.732921 23557 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115480851507-30472-0/minicluster-data/ts-1/data/info.pb
Aug 07 15:15:28 dist-test-slave-42z9 krb5kdc[23178](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115728, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.194.2@KRBTEST.COM for kudu/127.29.194.62@KRBTEST.COM
I20260807 15:15:28.742259 23318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:28.733632 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.194.2:58361 (local address 127.29.194.62:37251)
0807 15:15:28.733714 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:28.733716 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:28.733896 (+ 180us) server_negotiation.cc:407] Connection header received
0807 15:15:28.734386 (+ 490us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:28.734387 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:28.734398 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:28.734454 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:15:28.735413 (+ 959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:28.735923 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:28.736910 (+ 987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:28.737107 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:28.737164 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:28.739442 (+ 2278us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:15:28.739444 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:15:28.739444 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:15:28.739451 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:15:28.740861 (+ 1410us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:15:28.741305 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:15:28.741307 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:15:28.741307 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:15:28.741369 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:15:28.741706 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:15:28.741707 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:15:28.741707 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:15:28.741834 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:15:28.741895 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0807 15:15:28.742114 (+ 219us) server_negotiation.cc:299] Negotiation successful
0807 15:15:28.742190 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260807 15:15:28.742592 23747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:28.733608 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:37251 (local address 127.29.194.2:58361)
0807 15:15:28.733774 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:28.733812 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0807 15:15:28.734296 (+ 484us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:15:28.734512 (+ 216us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:15:28.734515 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:15:28.735098 (+ 583us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:15:28.735300 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:28.735304 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:28.736008 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:28.736008 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:28.736825 (+ 817us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:28.736827 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:28.737171 (+ 344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:28.737172 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:28.737270 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:28.737794 (+ 524us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:15:28.737803 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:15:28.739280 (+ 1477us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:15:28.740927 (+ 1647us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:15:28.740936 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:15:28.740937 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:15:28.741213 (+ 276us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:15:28.741475 (+ 262us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:15:28.741476 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:15:28.741476 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:15:28.741632 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:15:28.741890 (+ 258us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:15:28.741893 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:15:28.742014 (+ 121us) client_negotiation.cc:769] Sending connection context
0807 15:15:28.742095 (+ 81us) client_negotiation.cc:240] Negotiation successful
0807 15:15:28.742230 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":86,"threads_started":1}
I20260807 15:15:28.742974 30472 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7fZSIL/build/release/bin/kudu as pid 23557
I20260807 15:15:28.743100 30472 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115480851507-30472-0/minicluster-data/ts-1/wal/instance
I20260807 15:15:28.743189 23744 heartbeater.cc:347] Connected to a master server at 127.29.194.62:37251
I20260807 15:15:28.743302 23744 heartbeater.cc:464] Registering TS with master...
I20260807 15:15:28.743323 30472 external_mini_cluster.cc:1428] Setting key c0a95c9b82f31b8c837cfc6e49545248
I20260807 15:15:28.743510 23744 heartbeater.cc:511] Master 127.29.194.62:37251 requested a full tablet report, sending...
I20260807 15:15:28.744105 23257 ts_manager.cc:194] Registered new tserver with Master: 3a06c83f62054947a37122c46d1206be (127.29.194.2:33907)
I20260807 15:15:28.744727 23257 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.194.2@KRBTEST.COM'} at 127.29.194.2:58361
WARNING: no policy specified for kudu/127.29.194.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.194.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.194.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.3.keytab.
Entry for principal kudu/127.29.194.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.3.keytab.
I20260807 15:15:28.772212 30472 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7fZSIL/build/release/bin/kudu
/tmp/dist-test-task7fZSIL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115480851507-30472-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115480851507-30472-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115480851507-30472-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115480851507-30472-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.194.3:0
--local_ip_for_outbound_sockets=127.29.194.3
--webserver_interface=127.29.194.3
--webserver_port=0
--tserver_master_addrs=127.29.194.62:37251
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.3.keytab
--principal=kudu/127.29.194.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.194.20:46859
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115480851507-30472-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115480851507-30472-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115480851507-30472-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:15:28.856367 23752 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:15:28.856595 23752 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:15:28.856621 23752 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:15:28.857795 23752 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:15:28.857836 23752 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:15:28.857862 23752 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.194.3
I20260807 15:15:28.859313 23752 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.194.20:46859
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115480851507-30472-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115480851507-30472-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.3.keytab
--principal=kudu/127.29.194.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.194.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115480851507-30472-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.194.3
--webserver_port=0
--tserver_master_addrs=127.29.194.62:37251
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23752
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.194.3
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115480851507-30472-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:15:28.859567 23752 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:15:28.859790 23752 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:15:28.864646 23765 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:15:28.864658 23761 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:15:28.864836 23752 server_base.cc:1034] running on GCE node
W20260807 15:15:28.864730 23760 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:15:28.865254 23752 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:15:28.865573 23752 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:15:28.866708 23752 hybrid_clock.cc:648] HybridClock initialized: now 1786115728866689 us; error 35 us; skew 500 ppm
Aug 07 15:15:28 dist-test-slave-42z9 krb5kdc[23178](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115728, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.194.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:15:28.869168 23752 init.cc:377] Logged in from keytab as kudu/127.29.194.3@KRBTEST.COM (short username kudu)
I20260807 15:15:28.869628 23752 webserver.cc:460] Webserver started at http://127.29.194.3:46539/ using document root <none> and password file <none>
I20260807 15:15:28.869850 23752 fs_manager.cc:359] Metadata directory not provided
I20260807 15:15:28.869904 23752 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:15:28.870030 23752 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:15:28.871052 23752 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115480851507-30472-0/minicluster-data/ts-2/data/instance:
uuid: "7eb6f2ebb7644884b5a30b326def32a9"
format_stamp: "Formatted at 2026-08-07 15:15:28 on dist-test-slave-42z9"
server_key: "075bd74062011136364d080cd6db931c"
server_key_iv: "5986e2be3af660c8acf2225f843268db"
server_key_version: "encryptionkey@0"
I20260807 15:15:28.871388 23752 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115480851507-30472-0/minicluster-data/ts-2/wal/instance:
uuid: "7eb6f2ebb7644884b5a30b326def32a9"
format_stamp: "Formatted at 2026-08-07 15:15:28 on dist-test-slave-42z9"
server_key: "075bd74062011136364d080cd6db931c"
server_key_iv: "5986e2be3af660c8acf2225f843268db"
server_key_version: "encryptionkey@0"
I20260807 15:15:28.872691 23752 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 15:15:28.873416 23774 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:15:28.873605 23752 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 15:15:28.873687 23752 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115480851507-30472-0/minicluster-data/ts-2/data,/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115480851507-30472-0/minicluster-data/ts-2/wal
uuid: "7eb6f2ebb7644884b5a30b326def32a9"
format_stamp: "Formatted at 2026-08-07 15:15:28 on dist-test-slave-42z9"
server_key: "075bd74062011136364d080cd6db931c"
server_key_iv: "5986e2be3af660c8acf2225f843268db"
server_key_version: "encryptionkey@0"
I20260807 15:15:28.873754 23752 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115480851507-30472-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115480851507-30472-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115480851507-30472-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:15:28.883977 23752 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:15:28.886279 23752 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:15:28.886427 23752 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:15:28.886670 23752 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:15:28.887001 23752 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:15:28.887038 23752 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:15:28.887068 23752 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:15:28.887092 23752 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:15:28.893052 23752 rpc_server.cc:307] RPC server started. Bound to: 127.29.194.3:45553
I20260807 15:15:28.893121 23958 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.194.3:45553 every 8 connection(s)
I20260807 15:15:28.893428 23752 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115480851507-30472-0/minicluster-data/ts-2/data/info.pb
I20260807 15:15:28.898602 30472 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7fZSIL/build/release/bin/kudu as pid 23752
I20260807 15:15:28.898710 30472 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115480851507-30472-0/minicluster-data/ts-2/wal/instance
I20260807 15:15:28.898875 30472 external_mini_cluster.cc:1428] Setting key 2d71fd6a482b3b1c1c672226fcf1b936
Aug 07 15:15:28 dist-test-slave-42z9 krb5kdc[23178](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115728, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.194.3@KRBTEST.COM for kudu/127.29.194.62@KRBTEST.COM
I20260807 15:15:28.903465 23318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:28.894170 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.194.3:58091 (local address 127.29.194.62:37251)
0807 15:15:28.894222 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:28.894224 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:28.894415 (+ 191us) server_negotiation.cc:407] Connection header received
0807 15:15:28.894996 (+ 581us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:28.894998 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:28.895018 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:28.895072 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:15:28.896063 (+ 991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:28.896560 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:28.897565 (+ 1005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:28.897754 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:28.897809 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:28.900112 (+ 2303us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:15:28.900113 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:15:28.900114 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:15:28.900121 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:15:28.901763 (+ 1642us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:15:28.902290 (+ 527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:15:28.902292 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:15:28.902293 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:15:28.902385 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:15:28.902742 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:15:28.902743 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:15:28.902744 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:15:28.902894 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:15:28.902965 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0807 15:15:28.903267 (+ 302us) server_negotiation.cc:299] Negotiation successful
0807 15:15:28.903376 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260807 15:15:28.903520 23961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:28.894132 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:37251 (local address 127.29.194.3:58091)
0807 15:15:28.894293 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:28.894332 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0807 15:15:28.894889 (+ 557us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:15:28.895147 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:15:28.895152 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:15:28.895754 (+ 602us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:15:28.895956 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:28.895961 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:28.896652 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:28.896653 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:28.897483 (+ 830us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:28.897486 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:28.897826 (+ 340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:28.897828 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:28.897967 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:28.898529 (+ 562us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:15:28.898538 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:15:28.900010 (+ 1472us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:15:28.901914 (+ 1904us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:15:28.901923 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:15:28.901926 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:15:28.902167 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:15:28.902498 (+ 331us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:15:28.902499 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:15:28.902499 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:15:28.902635 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:15:28.902960 (+ 325us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:15:28.902964 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:15:28.903088 (+ 124us) client_negotiation.cc:769] Sending connection context
0807 15:15:28.903174 (+ 86us) client_negotiation.cc:240] Negotiation successful
0807 15:15:28.903317 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":71,"threads_started":1}
I20260807 15:15:28.904170 23959 heartbeater.cc:347] Connected to a master server at 127.29.194.62:37251
I20260807 15:15:28.904271 23959 heartbeater.cc:464] Registering TS with master...
I20260807 15:15:28.904742 23959 heartbeater.cc:511] Master 127.29.194.62:37251 requested a full tablet report, sending...
I20260807 15:15:28.905186 23257 ts_manager.cc:194] Registered new tserver with Master: 7eb6f2ebb7644884b5a30b326def32a9 (127.29.194.3:45553)
I20260807 15:15:28.905679 23257 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.194.3@KRBTEST.COM'} at 127.29.194.3:58091
I20260807 15:15:28.910900 30472 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 15:15:28.918324 23318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:28.912278 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33738 (local address 127.29.194.62:37251)
0807 15:15:28.912334 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:28.912336 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:28.912421 (+ 85us) server_negotiation.cc:407] Connection header received
0807 15:15:28.912466 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:28.912467 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:28.912478 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:28.912530 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:15:28.913047 (+ 517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:28.913579 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:28.914295 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:28.914480 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:28.914551 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:28.915901 (+ 1350us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:15:28.915902 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:15:28.915903 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:15:28.915910 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:15:28.917279 (+ 1369us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:15:28.917692 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:15:28.917693 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:15:28.917693 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:15:28.917745 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:15:28.917882 (+ 137us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:15:28.917882 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:15:28.917882 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:15:28.918034 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:15:28.918102 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0807 15:15:28.918183 (+ 81us) server_negotiation.cc:299] Negotiation successful
0807 15:15:28.918249 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260807 15:15:28.919922 23257 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33738:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 15:15:28.921744 23257 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 15:15:28.928690 23971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:28.924607 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.2:33907 (local address 127.0.0.1:45158)
0807 15:15:28.925030 (+ 423us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:28.925075 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0807 15:15:28.925150 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:15:28.925496 (+ 346us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:15:28.925497 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:15:28.925498 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:15:28.925555 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:28.925556 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:28.926569 (+ 1013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:28.926570 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:28.927648 (+ 1078us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:28.927650 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:28.928346 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:28.928347 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:28.928444 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:28.928460 (+ 16us) client_negotiation.cc:769] Sending connection context
0807 15:15:28.928539 (+ 79us) client_negotiation.cc:240] Negotiation successful
0807 15:15:28.928600 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":393,"mutex_wait_us":3,"thread_start_us":49,"threads_started":1}
I20260807 15:15:28.928694 23976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:28.924799 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45158 (local address 127.29.194.2:33907)
0807 15:15:28.924971 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:28.924976 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:28.925234 (+ 258us) server_negotiation.cc:407] Connection header received
0807 15:15:28.925282 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:28.925285 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:28.925300 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:28.925379 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:15:28.925650 (+ 271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:28.926475 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:28.927725 (+ 1250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:28.928262 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:28.928333 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:28.928397 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0807 15:15:28.928560 (+ 163us) server_negotiation.cc:299] Negotiation successful
0807 15:15:28.928606 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":134,"thread_start_us":95,"threads_started":1}
I20260807 15:15:28.929306 23978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:28.924607 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.3:45553 (local address 127.0.0.1:49946)
0807 15:15:28.925030 (+ 423us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:28.925086 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0807 15:15:28.925149 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:15:28.925720 (+ 571us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:15:28.925721 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:15:28.925723 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:15:28.925754 (+ 31us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:28.925755 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:28.926948 (+ 1193us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:28.926950 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:28.928162 (+ 1212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:28.928165 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:28.929052 (+ 887us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:28.929054 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:28.929130 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:28.929133 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 15:15:28.929194 (+ 61us) client_negotiation.cc:240] Negotiation successful
0807 15:15:28.929236 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":397,"mutex_wait_us":5,"thread_start_us":90,"threads_started":1}
I20260807 15:15:28.929338 23974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:28.924742 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49946 (local address 127.29.194.3:45553)
0807 15:15:28.925565 (+ 823us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:28.925569 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:28.925586 (+ 17us) server_negotiation.cc:407] Connection header received
0807 15:15:28.925625 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:28.925627 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:28.925642 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:28.925723 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:15:28.925801 (+ 78us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:28.926866 (+ 1065us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:28.928265 (+ 1399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:28.928969 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:28.929045 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:28.929109 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0807 15:15:28.929215 (+ 106us) server_negotiation.cc:299] Negotiation successful
0807 15:15:28.929262 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":789,"thread_start_us":90,"threads_started":1}
I20260807 15:15:28.929740 23972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:28.924797 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.1:42461 (local address 127.0.0.1:41588)
0807 15:15:28.925034 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:28.925075 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0807 15:15:28.925203 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:15:28.925432 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:15:28.925435 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:15:28.925459 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:15:28.925548 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:28.925550 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:28.926983 (+ 1433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:28.926984 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:28.928168 (+ 1184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:28.928170 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:28.929564 (+ 1394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:28.929565 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:28.929626 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:28.929629 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 15:15:28.929663 (+ 34us) client_negotiation.cc:240] Negotiation successful
0807 15:15:28.929695 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":211,"mutex_wait_us":74,"thread_start_us":106,"threads_started":1}
I20260807 15:15:28.929750 23649 tablet_service.cc:1467] Processing CreateTablet for tablet 71efcd73a5b4470d82cdf5eda3a70cbb (DEFAULT_TABLE table=test-table [id=e394a8f85cac4b6c891dc6a0c2ae34d1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:15:28.930160 23649 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 71efcd73a5b4470d82cdf5eda3a70cbb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:15:28.930158 23979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:28.924921 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41588 (local address 127.29.194.1:42461)
0807 15:15:28.925119 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:28.925123 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:28.925134 (+ 11us) server_negotiation.cc:407] Connection header received
0807 15:15:28.925289 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:28.925291 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:28.925306 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:28.925380 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:15:28.925629 (+ 249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:28.926791 (+ 1162us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:28.928288 (+ 1497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:28.929265 (+ 977us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:28.929323 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:28.929997 (+ 674us) server_negotiation.cc:1035] Waiting for connection context
0807 15:15:28.930025 (+ 28us) server_negotiation.cc:299] Negotiation successful
0807 15:15:28.930077 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":167,"thread_start_us":91,"threads_started":1}
I20260807 15:15:28.930225 23854 tablet_service.cc:1467] Processing CreateTablet for tablet 71efcd73a5b4470d82cdf5eda3a70cbb (DEFAULT_TABLE table=test-table [id=e394a8f85cac4b6c891dc6a0c2ae34d1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:15:28.930593 23854 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 71efcd73a5b4470d82cdf5eda3a70cbb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:15:28.930862 23445 tablet_service.cc:1467] Processing CreateTablet for tablet 71efcd73a5b4470d82cdf5eda3a70cbb (DEFAULT_TABLE table=test-table [id=e394a8f85cac4b6c891dc6a0c2ae34d1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:15:28.931257 23445 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 71efcd73a5b4470d82cdf5eda3a70cbb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:15:28.933308 23982 tablet_bootstrap.cc:492] T 71efcd73a5b4470d82cdf5eda3a70cbb P 7eb6f2ebb7644884b5a30b326def32a9: Bootstrap starting.
I20260807 15:15:28.933799 23983 tablet_bootstrap.cc:492] T 71efcd73a5b4470d82cdf5eda3a70cbb P 3a06c83f62054947a37122c46d1206be: Bootstrap starting.
I20260807 15:15:28.934170 23982 tablet_bootstrap.cc:654] T 71efcd73a5b4470d82cdf5eda3a70cbb P 7eb6f2ebb7644884b5a30b326def32a9: Neither blocks nor log segments found. Creating new log.
I20260807 15:15:28.934463 23983 tablet_bootstrap.cc:654] T 71efcd73a5b4470d82cdf5eda3a70cbb P 3a06c83f62054947a37122c46d1206be: Neither blocks nor log segments found. Creating new log.
I20260807 15:15:28.934640 23987 tablet_bootstrap.cc:492] T 71efcd73a5b4470d82cdf5eda3a70cbb P 2a50a320ccb04b85bbb55d30db39721f: Bootstrap starting.
I20260807 15:15:28.934687 23982 log.cc:824] T 71efcd73a5b4470d82cdf5eda3a70cbb P 7eb6f2ebb7644884b5a30b326def32a9: Log is configured to *not* fsync() on all Append() calls
I20260807 15:15:28.935029 23983 log.cc:824] T 71efcd73a5b4470d82cdf5eda3a70cbb P 3a06c83f62054947a37122c46d1206be: Log is configured to *not* fsync() on all Append() calls
I20260807 15:15:28.935277 23987 tablet_bootstrap.cc:654] T 71efcd73a5b4470d82cdf5eda3a70cbb P 2a50a320ccb04b85bbb55d30db39721f: Neither blocks nor log segments found. Creating new log.
I20260807 15:15:28.935510 23982 tablet_bootstrap.cc:492] T 71efcd73a5b4470d82cdf5eda3a70cbb P 7eb6f2ebb7644884b5a30b326def32a9: No bootstrap required, opened a new log
I20260807 15:15:28.935591 23982 ts_tablet_manager.cc:1397] T 71efcd73a5b4470d82cdf5eda3a70cbb P 7eb6f2ebb7644884b5a30b326def32a9: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260807 15:15:28.935801 23987 log.cc:824] T 71efcd73a5b4470d82cdf5eda3a70cbb P 2a50a320ccb04b85bbb55d30db39721f: Log is configured to *not* fsync() on all Append() calls
I20260807 15:15:28.936084 23983 tablet_bootstrap.cc:492] T 71efcd73a5b4470d82cdf5eda3a70cbb P 3a06c83f62054947a37122c46d1206be: No bootstrap required, opened a new log
I20260807 15:15:28.936156 23983 ts_tablet_manager.cc:1397] T 71efcd73a5b4470d82cdf5eda3a70cbb P 3a06c83f62054947a37122c46d1206be: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260807 15:15:28.936419 23987 tablet_bootstrap.cc:492] T 71efcd73a5b4470d82cdf5eda3a70cbb P 2a50a320ccb04b85bbb55d30db39721f: No bootstrap required, opened a new log
I20260807 15:15:28.936482 23987 ts_tablet_manager.cc:1397] T 71efcd73a5b4470d82cdf5eda3a70cbb P 2a50a320ccb04b85bbb55d30db39721f: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 15:15:28.937624 23982 raft_consensus.cc:348] T 71efcd73a5b4470d82cdf5eda3a70cbb P 7eb6f2ebb7644884b5a30b326def32a9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eb6f2ebb7644884b5a30b326def32a9" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 45553 } } peers { permanent_uuid: "3a06c83f62054947a37122c46d1206be" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 33907 } } peers { permanent_uuid: "2a50a320ccb04b85bbb55d30db39721f" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 42461 } }
I20260807 15:15:28.937826 23982 raft_consensus.cc:374] T 71efcd73a5b4470d82cdf5eda3a70cbb P 7eb6f2ebb7644884b5a30b326def32a9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:15:28.937868 23982 raft_consensus.cc:729] T 71efcd73a5b4470d82cdf5eda3a70cbb P 7eb6f2ebb7644884b5a30b326def32a9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7eb6f2ebb7644884b5a30b326def32a9, State: Initialized, Role: FOLLOWER
I20260807 15:15:28.937995 23982 consensus_queue.cc:260] T 71efcd73a5b4470d82cdf5eda3a70cbb P 7eb6f2ebb7644884b5a30b326def32a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eb6f2ebb7644884b5a30b326def32a9" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 45553 } } peers { permanent_uuid: "3a06c83f62054947a37122c46d1206be" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 33907 } } peers { permanent_uuid: "2a50a320ccb04b85bbb55d30db39721f" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 42461 } }
I20260807 15:15:28.938288 23982 ts_tablet_manager.cc:1428] T 71efcd73a5b4470d82cdf5eda3a70cbb P 7eb6f2ebb7644884b5a30b326def32a9: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260807 15:15:28.938205 23983 raft_consensus.cc:348] T 71efcd73a5b4470d82cdf5eda3a70cbb P 3a06c83f62054947a37122c46d1206be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eb6f2ebb7644884b5a30b326def32a9" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 45553 } } peers { permanent_uuid: "3a06c83f62054947a37122c46d1206be" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 33907 } } peers { permanent_uuid: "2a50a320ccb04b85bbb55d30db39721f" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 42461 } }
I20260807 15:15:28.938297 23959 heartbeater.cc:503] Master 127.29.194.62:37251 was elected leader, sending a full tablet report...
I20260807 15:15:28.938361 23983 raft_consensus.cc:374] T 71efcd73a5b4470d82cdf5eda3a70cbb P 3a06c83f62054947a37122c46d1206be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:15:28.938396 23983 raft_consensus.cc:729] T 71efcd73a5b4470d82cdf5eda3a70cbb P 3a06c83f62054947a37122c46d1206be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a06c83f62054947a37122c46d1206be, State: Initialized, Role: FOLLOWER
I20260807 15:15:28.938499 23983 consensus_queue.cc:260] T 71efcd73a5b4470d82cdf5eda3a70cbb P 3a06c83f62054947a37122c46d1206be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eb6f2ebb7644884b5a30b326def32a9" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 45553 } } peers { permanent_uuid: "3a06c83f62054947a37122c46d1206be" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 33907 } } peers { permanent_uuid: "2a50a320ccb04b85bbb55d30db39721f" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 42461 } }
I20260807 15:15:28.938436 23987 raft_consensus.cc:348] T 71efcd73a5b4470d82cdf5eda3a70cbb P 2a50a320ccb04b85bbb55d30db39721f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eb6f2ebb7644884b5a30b326def32a9" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 45553 } } peers { permanent_uuid: "3a06c83f62054947a37122c46d1206be" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 33907 } } peers { permanent_uuid: "2a50a320ccb04b85bbb55d30db39721f" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 42461 } }
I20260807 15:15:28.938602 23987 raft_consensus.cc:374] T 71efcd73a5b4470d82cdf5eda3a70cbb P 2a50a320ccb04b85bbb55d30db39721f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:15:28.938647 23987 raft_consensus.cc:729] T 71efcd73a5b4470d82cdf5eda3a70cbb P 2a50a320ccb04b85bbb55d30db39721f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a50a320ccb04b85bbb55d30db39721f, State: Initialized, Role: FOLLOWER
I20260807 15:15:28.938756 23983 ts_tablet_manager.cc:1428] T 71efcd73a5b4470d82cdf5eda3a70cbb P 3a06c83f62054947a37122c46d1206be: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260807 15:15:28.938747 23987 consensus_queue.cc:260] T 71efcd73a5b4470d82cdf5eda3a70cbb P 2a50a320ccb04b85bbb55d30db39721f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eb6f2ebb7644884b5a30b326def32a9" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 45553 } } peers { permanent_uuid: "3a06c83f62054947a37122c46d1206be" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 33907 } } peers { permanent_uuid: "2a50a320ccb04b85bbb55d30db39721f" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 42461 } }
I20260807 15:15:28.938877 23744 heartbeater.cc:503] Master 127.29.194.62:37251 was elected leader, sending a full tablet report...
I20260807 15:15:28.939005 23987 ts_tablet_manager.cc:1428] T 71efcd73a5b4470d82cdf5eda3a70cbb P 2a50a320ccb04b85bbb55d30db39721f: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260807 15:15:28.939035 23549 heartbeater.cc:503] Master 127.29.194.62:37251 was elected leader, sending a full tablet report...
W20260807 15:15:28.983574 23746 tablet.cc:2369] T 71efcd73a5b4470d82cdf5eda3a70cbb P 3a06c83f62054947a37122c46d1206be: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 15:15:29.068915 23551 tablet.cc:2369] T 71efcd73a5b4470d82cdf5eda3a70cbb P 2a50a320ccb04b85bbb55d30db39721f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 15:15:29.129717 23993 raft_consensus.cc:482] T 71efcd73a5b4470d82cdf5eda3a70cbb P 2a50a320ccb04b85bbb55d30db39721f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 15:15:29.129855 23993 raft_consensus.cc:504] T 71efcd73a5b4470d82cdf5eda3a70cbb P 2a50a320ccb04b85bbb55d30db39721f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eb6f2ebb7644884b5a30b326def32a9" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 45553 } } peers { permanent_uuid: "3a06c83f62054947a37122c46d1206be" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 33907 } } peers { permanent_uuid: "2a50a320ccb04b85bbb55d30db39721f" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 42461 } }
I20260807 15:15:29.130264 23993 leader_election.cc:288] T 71efcd73a5b4470d82cdf5eda3a70cbb P 2a50a320ccb04b85bbb55d30db39721f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7eb6f2ebb7644884b5a30b326def32a9 (127.29.194.3:45553), 3a06c83f62054947a37122c46d1206be (127.29.194.2:33907)
I20260807 15:15:29.134730 23976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:29.130562 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.194.1:38775 (local address 127.29.194.2:33907)
0807 15:15:29.130614 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:29.130616 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:29.130654 (+ 38us) server_negotiation.cc:407] Connection header received
0807 15:15:29.130868 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:29.130869 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:29.130876 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:29.130921 (+ 45us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:15:29.131060 (+ 139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:29.131753 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:29.133432 (+ 1679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:29.134274 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:29.134372 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:29.134383 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0807 15:15:29.134600 (+ 217us) server_negotiation.cc:299] Negotiation successful
0807 15:15:29.134650 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260807 15:15:29.134730 23974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:29.130675 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.194.1:51583 (local address 127.29.194.3:45553)
0807 15:15:29.130720 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:29.130721 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:29.130729 (+ 8us) server_negotiation.cc:407] Connection header received
0807 15:15:29.130754 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:29.130755 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:29.130763 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:29.130815 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:15:29.131002 (+ 187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:29.131755 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:29.133431 (+ 1676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:29.134294 (+ 863us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:29.134372 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:29.134383 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0807 15:15:29.134600 (+ 217us) server_negotiation.cc:299] Negotiation successful
0807 15:15:29.134651 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260807 15:15:29.134768 23994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:29.130465 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.2:33907 (local address 127.29.194.1:38775)
0807 15:15:29.130579 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:29.130590 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 15:15:29.130667 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:15:29.130931 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:15:29.130931 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:15:29.130933 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:15:29.130998 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:29.131000 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:29.131865 (+ 865us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:29.131866 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:29.133225 (+ 1359us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:29.133230 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:29.134398 (+ 1168us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:29.134399 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:29.134512 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:29.134522 (+ 10us) client_negotiation.cc:769] Sending connection context
0807 15:15:29.134592 (+ 70us) client_negotiation.cc:240] Negotiation successful
0807 15:15:29.134654 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":93,"mutex_wait_us":10,"thread_start_us":86,"threads_started":1}
I20260807 15:15:29.134770 23995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:29.130468 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.3:45553 (local address 127.29.194.1:51583)
0807 15:15:29.130627 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:29.130635 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0807 15:15:29.130676 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:15:29.130828 (+ 152us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:15:29.130829 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:15:29.130831 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:15:29.130913 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:29.130915 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:29.131865 (+ 950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:29.131866 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:29.133225 (+ 1359us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:29.133230 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:29.134396 (+ 1166us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:29.134397 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:29.134512 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:29.134521 (+ 9us) client_negotiation.cc:769] Sending connection context
0807 15:15:29.134592 (+ 71us) client_negotiation.cc:240] Negotiation successful
0807 15:15:29.134654 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"mutex_wait_us":125,"thread_start_us":63,"threads_started":1}
I20260807 15:15:29.135154 23677 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "71efcd73a5b4470d82cdf5eda3a70cbb" candidate_uuid: "2a50a320ccb04b85bbb55d30db39721f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a06c83f62054947a37122c46d1206be" is_pre_election: true
I20260807 15:15:29.135407 23677 raft_consensus.cc:2454] T 71efcd73a5b4470d82cdf5eda3a70cbb P 3a06c83f62054947a37122c46d1206be [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2a50a320ccb04b85bbb55d30db39721f in term 0.
I20260807 15:15:29.135476 23887 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "71efcd73a5b4470d82cdf5eda3a70cbb" candidate_uuid: "2a50a320ccb04b85bbb55d30db39721f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7eb6f2ebb7644884b5a30b326def32a9" is_pre_election: true
I20260807 15:15:29.135641 23887 raft_consensus.cc:2454] T 71efcd73a5b4470d82cdf5eda3a70cbb P 7eb6f2ebb7644884b5a30b326def32a9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2a50a320ccb04b85bbb55d30db39721f in term 0.
I20260807 15:15:29.135937 23377 leader_election.cc:302] T 71efcd73a5b4470d82cdf5eda3a70cbb P 2a50a320ccb04b85bbb55d30db39721f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2a50a320ccb04b85bbb55d30db39721f, 3a06c83f62054947a37122c46d1206be; no voters:
I20260807 15:15:29.136205 23993 raft_consensus.cc:2781] T 71efcd73a5b4470d82cdf5eda3a70cbb P 2a50a320ccb04b85bbb55d30db39721f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 15:15:29.136292 23993 raft_consensus.cc:482] T 71efcd73a5b4470d82cdf5eda3a70cbb P 2a50a320ccb04b85bbb55d30db39721f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 15:15:29.136317 23993 raft_consensus.cc:3037] T 71efcd73a5b4470d82cdf5eda3a70cbb P 2a50a320ccb04b85bbb55d30db39721f [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:15:29.137099 23993 raft_consensus.cc:504] T 71efcd73a5b4470d82cdf5eda3a70cbb P 2a50a320ccb04b85bbb55d30db39721f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eb6f2ebb7644884b5a30b326def32a9" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 45553 } } peers { permanent_uuid: "3a06c83f62054947a37122c46d1206be" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 33907 } } peers { permanent_uuid: "2a50a320ccb04b85bbb55d30db39721f" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 42461 } }
I20260807 15:15:29.137262 23993 leader_election.cc:288] T 71efcd73a5b4470d82cdf5eda3a70cbb P 2a50a320ccb04b85bbb55d30db39721f [CANDIDATE]: Term 1 election: Requested vote from peers 7eb6f2ebb7644884b5a30b326def32a9 (127.29.194.3:45553), 3a06c83f62054947a37122c46d1206be (127.29.194.2:33907)
I20260807 15:15:29.137531 23887 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "71efcd73a5b4470d82cdf5eda3a70cbb" candidate_uuid: "2a50a320ccb04b85bbb55d30db39721f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7eb6f2ebb7644884b5a30b326def32a9"
I20260807 15:15:29.137643 23677 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "71efcd73a5b4470d82cdf5eda3a70cbb" candidate_uuid: "2a50a320ccb04b85bbb55d30db39721f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a06c83f62054947a37122c46d1206be"
I20260807 15:15:29.137665 23887 raft_consensus.cc:3037] T 71efcd73a5b4470d82cdf5eda3a70cbb P 7eb6f2ebb7644884b5a30b326def32a9 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:15:29.137723 23677 raft_consensus.cc:3037] T 71efcd73a5b4470d82cdf5eda3a70cbb P 3a06c83f62054947a37122c46d1206be [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:15:29.138643 23677 raft_consensus.cc:2454] T 71efcd73a5b4470d82cdf5eda3a70cbb P 3a06c83f62054947a37122c46d1206be [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2a50a320ccb04b85bbb55d30db39721f in term 1.
I20260807 15:15:29.138644 23887 raft_consensus.cc:2454] T 71efcd73a5b4470d82cdf5eda3a70cbb P 7eb6f2ebb7644884b5a30b326def32a9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2a50a320ccb04b85bbb55d30db39721f in term 1.
I20260807 15:15:29.138885 23379 leader_election.cc:302] T 71efcd73a5b4470d82cdf5eda3a70cbb P 2a50a320ccb04b85bbb55d30db39721f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2a50a320ccb04b85bbb55d30db39721f, 7eb6f2ebb7644884b5a30b326def32a9; no voters:
I20260807 15:15:29.138993 23993 raft_consensus.cc:2781] T 71efcd73a5b4470d82cdf5eda3a70cbb P 2a50a320ccb04b85bbb55d30db39721f [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:15:29.139163 23993 raft_consensus.cc:686] T 71efcd73a5b4470d82cdf5eda3a70cbb P 2a50a320ccb04b85bbb55d30db39721f [term 1 LEADER]: Becoming Leader. State: Replica: 2a50a320ccb04b85bbb55d30db39721f, State: Running, Role: LEADER
I20260807 15:15:29.139281 23993 consensus_queue.cc:237] T 71efcd73a5b4470d82cdf5eda3a70cbb P 2a50a320ccb04b85bbb55d30db39721f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eb6f2ebb7644884b5a30b326def32a9" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 45553 } } peers { permanent_uuid: "3a06c83f62054947a37122c46d1206be" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 33907 } } peers { permanent_uuid: "2a50a320ccb04b85bbb55d30db39721f" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 42461 } }
I20260807 15:15:29.140162 23256 catalog_manager.cc:5495] T 71efcd73a5b4470d82cdf5eda3a70cbb P 2a50a320ccb04b85bbb55d30db39721f reported cstate change: term changed from 0 to 1, leader changed from <none> to 2a50a320ccb04b85bbb55d30db39721f (127.29.194.1). New cstate: current_term: 1 leader_uuid: "2a50a320ccb04b85bbb55d30db39721f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eb6f2ebb7644884b5a30b326def32a9" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 45553 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3a06c83f62054947a37122c46d1206be" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 33907 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2a50a320ccb04b85bbb55d30db39721f" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 42461 } health_report { overall_health: HEALTHY } } }
W20260807 15:15:29.144032 23960 tablet.cc:2369] T 71efcd73a5b4470d82cdf5eda3a70cbb P 7eb6f2ebb7644884b5a30b326def32a9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 15:15:29.158993 23979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:29.155579 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41602 (local address 127.29.194.1:42461)
0807 15:15:29.155634 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:29.155636 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:29.155647 (+ 11us) server_negotiation.cc:407] Connection header received
0807 15:15:29.155695 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:29.155696 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:29.155716 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:29.155804 (+ 88us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 15:15:29.155956 (+ 152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:29.156700 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:29.157956 (+ 1256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:29.158253 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:29.158334 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:29.158560 (+ 226us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 15:15:29.158721 (+ 161us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 15:15:29.158814 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0807 15:15:29.158869 (+ 55us) server_negotiation.cc:299] Negotiation successful
0807 15:15:29.158901 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260807 15:15:29.161177 23887 raft_consensus.cc:1261] T 71efcd73a5b4470d82cdf5eda3a70cbb P 7eb6f2ebb7644884b5a30b326def32a9 [term 1 FOLLOWER]: Refusing update from remote peer 2a50a320ccb04b85bbb55d30db39721f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:15:29.161175 23677 raft_consensus.cc:1261] T 71efcd73a5b4470d82cdf5eda3a70cbb P 3a06c83f62054947a37122c46d1206be [term 1 FOLLOWER]: Refusing update from remote peer 2a50a320ccb04b85bbb55d30db39721f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:15:29.161568 23997 consensus_queue.cc:1035] T 71efcd73a5b4470d82cdf5eda3a70cbb P 2a50a320ccb04b85bbb55d30db39721f [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eb6f2ebb7644884b5a30b326def32a9" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:15:29.161717 23993 consensus_queue.cc:1035] T 71efcd73a5b4470d82cdf5eda3a70cbb P 2a50a320ccb04b85bbb55d30db39721f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a06c83f62054947a37122c46d1206be" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 33907 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:15:29.163949 24001 mvcc.cc:205] Tried to move back new op lower bound from 7315930026641055744 to 7315930026555076608. Current Snapshot: MvccSnapshot[applied={T|T < 7315930026641055744}]
I20260807 15:15:29.164283 24003 mvcc.cc:205] Tried to move back new op lower bound from 7315930026641055744 to 7315930026555076608. Current Snapshot: MvccSnapshot[applied={T|T < 7315930026641055744}]
I20260807 15:15:29.164413 24002 mvcc.cc:205] Tried to move back new op lower bound from 7315930026641055744 to 7315930026555076608. Current Snapshot: MvccSnapshot[applied={T|T < 7315930026641055744}]
I20260807 15:15:29.167274 23256 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33738:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 15:15:29.167374 23256 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33738:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 15:15:29.168162 23256 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 832fb5572e1b48ab9521c052b74260c2: Sending DeleteTablet for 3 replicas of tablet 71efcd73a5b4470d82cdf5eda3a70cbb
I20260807 15:15:29.168604 23854 tablet_service.cc:1514] Processing DeleteTablet for tablet 71efcd73a5b4470d82cdf5eda3a70cbb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:15:29 UTC) from {username='kudu', principal='kudu/127.29.194.62@KRBTEST.COM'} at 127.0.0.1:49946
I20260807 15:15:29.168604 23445 tablet_service.cc:1514] Processing DeleteTablet for tablet 71efcd73a5b4470d82cdf5eda3a70cbb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:15:29 UTC) from {username='kudu', principal='kudu/127.29.194.62@KRBTEST.COM'} at 127.0.0.1:41588
I20260807 15:15:29.168875 23649 tablet_service.cc:1514] Processing DeleteTablet for tablet 71efcd73a5b4470d82cdf5eda3a70cbb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:15:29 UTC) from {username='kudu', principal='kudu/127.29.194.62@KRBTEST.COM'} at 127.0.0.1:45158
I20260807 15:15:29.169112 24017 tablet_replica.cc:330] T 71efcd73a5b4470d82cdf5eda3a70cbb P 3a06c83f62054947a37122c46d1206be: stopping tablet replica
I20260807 15:15:29.169127 30472 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7fZSIL/build/release/bin/kudu with pid 23352
I20260807 15:15:29.169175 24014 tablet_replica.cc:330] T 71efcd73a5b4470d82cdf5eda3a70cbb P 2a50a320ccb04b85bbb55d30db39721f: stopping tablet replica
I20260807 15:15:29.169275 24014 raft_consensus.cc:2229] T 71efcd73a5b4470d82cdf5eda3a70cbb P 2a50a320ccb04b85bbb55d30db39721f [term 1 LEADER]: Raft consensus shutting down.
I20260807 15:15:29.169399 24014 raft_consensus.cc:2258] T 71efcd73a5b4470d82cdf5eda3a70cbb P 2a50a320ccb04b85bbb55d30db39721f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:15:29.169586 24015 tablet_replica.cc:330] T 71efcd73a5b4470d82cdf5eda3a70cbb P 7eb6f2ebb7644884b5a30b326def32a9: stopping tablet replica
I20260807 15:15:29.169734 24015 raft_consensus.cc:2229] T 71efcd73a5b4470d82cdf5eda3a70cbb P 7eb6f2ebb7644884b5a30b326def32a9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 15:15:29.169837 24015 raft_consensus.cc:2258] T 71efcd73a5b4470d82cdf5eda3a70cbb P 7eb6f2ebb7644884b5a30b326def32a9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:15:29.176697 24017 raft_consensus.cc:2229] T 71efcd73a5b4470d82cdf5eda3a70cbb P 3a06c83f62054947a37122c46d1206be [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 15:15:29.176882 24017 raft_consensus.cc:2258] T 71efcd73a5b4470d82cdf5eda3a70cbb P 3a06c83f62054947a37122c46d1206be [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:15:29.178417 24017 ts_tablet_manager.cc:1905] T 71efcd73a5b4470d82cdf5eda3a70cbb P 3a06c83f62054947a37122c46d1206be: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:15:29.180414 24017 ts_tablet_manager.cc:1918] T 71efcd73a5b4470d82cdf5eda3a70cbb P 3a06c83f62054947a37122c46d1206be: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 15:15:29.180601 24017 log.cc:1196] T 71efcd73a5b4470d82cdf5eda3a70cbb P 3a06c83f62054947a37122c46d1206be: Deleting WAL directory at /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115480851507-30472-0/minicluster-data/ts-1/wal/wals/71efcd73a5b4470d82cdf5eda3a70cbb
I20260807 15:15:29.180689 24015 ts_tablet_manager.cc:1905] T 71efcd73a5b4470d82cdf5eda3a70cbb P 7eb6f2ebb7644884b5a30b326def32a9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:15:29.181015 24017 ts_tablet_manager.cc:1939] T 71efcd73a5b4470d82cdf5eda3a70cbb P 3a06c83f62054947a37122c46d1206be: Deleting consensus metadata
I20260807 15:15:29.181557 23234 catalog_manager.cc:4841] TS 3a06c83f62054947a37122c46d1206be (127.29.194.2:33907): tablet 71efcd73a5b4470d82cdf5eda3a70cbb (table test-table [id=e394a8f85cac4b6c891dc6a0c2ae34d1]) successfully deleted
I20260807 15:15:29.182106 24015 ts_tablet_manager.cc:1918] T 71efcd73a5b4470d82cdf5eda3a70cbb P 7eb6f2ebb7644884b5a30b326def32a9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 15:15:29.182170 24015 log.cc:1196] T 71efcd73a5b4470d82cdf5eda3a70cbb P 7eb6f2ebb7644884b5a30b326def32a9: Deleting WAL directory at /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115480851507-30472-0/minicluster-data/ts-2/wal/wals/71efcd73a5b4470d82cdf5eda3a70cbb
I20260807 15:15:29.182391 24015 ts_tablet_manager.cc:1939] T 71efcd73a5b4470d82cdf5eda3a70cbb P 7eb6f2ebb7644884b5a30b326def32a9: Deleting consensus metadata
I20260807 15:15:29.182847 23237 catalog_manager.cc:4841] TS 7eb6f2ebb7644884b5a30b326def32a9 (127.29.194.3:45553): tablet 71efcd73a5b4470d82cdf5eda3a70cbb (table test-table [id=e394a8f85cac4b6c891dc6a0c2ae34d1]) successfully deleted
W20260807 15:15:29.183370 23234 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.29.194.1:42461 (error 108)
I20260807 15:15:29.183569 30472 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7fZSIL/build/release/bin/kudu with pid 23557
I20260807 15:15:29.184278 23972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:29.184165 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.1:42461 (local address 127.0.0.1:41612)
0807 15:15:29.184212 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:29.184246 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.194.1:42461: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260807 15:15:29.184471 23234 catalog_manager.cc:4584] TS 2a50a320ccb04b85bbb55d30db39721f (127.29.194.1:42461): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 71efcd73a5b4470d82cdf5eda3a70cbb: Network error: Client connection negotiation failed: client connection to 127.29.194.1:42461: connect: Connection refused (error 111)
I20260807 15:15:29.197903 30472 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7fZSIL/build/release/bin/kudu with pid 23752
I20260807 15:15:29.217546 30472 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7fZSIL/build/release/bin/kudu with pid 23206
2026-08-07T15:15:29Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (2324 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-07T15:15:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T15:15:29Z Disabled control of system clock
I20260807 15:15:29.259649 30472 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7fZSIL/build/release/bin/kudu
/tmp/dist-test-task7fZSIL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115480851507-30472-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115480851507-30472-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115480851507-30472-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115480851507-30472-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.194.62:34763
--webserver_interface=127.29.194.62
--webserver_port=0
--builtin_ntp_servers=127.29.194.20:33799
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.194.62:34763
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260807 15:15:29.344040 24024 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:15:29.344300 24024 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:15:29.344322 24024 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:15:29.345595 24024 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:15:29.345646 24024 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:15:29.345659 24024 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:15:29.345670 24024 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:15:29.345681 24024 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:15:29.347139 24024 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.194.20:33799
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115480851507-30472-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115480851507-30472-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.194.62:34763
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.194.62:34763
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115480851507-30472-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.194.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24024
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115480851507-30472-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:15:29.347381 24024 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:15:29.347651 24024 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:15:29.352782 24033 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:15:29.352775 24036 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:15:29.352900 24024 server_base.cc:1034] running on GCE node
W20260807 15:15:29.352775 24032 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:15:29.353492 24024 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:15:29.353782 24024 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:15:29.354956 24024 hybrid_clock.cc:648] HybridClock initialized: now 1786115729354933 us; error 38 us; skew 500 ppm
I20260807 15:15:29.356667 24024 webserver.cc:460] Webserver started at http://127.29.194.62:37979/ using document root <none> and password file <none>
I20260807 15:15:29.357002 24024 fs_manager.cc:359] Metadata directory not provided
I20260807 15:15:29.357064 24024 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:15:29.357335 24024 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:15:29.358568 24024 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115480851507-30472-0/minicluster-data/master-0/data/instance:
uuid: "caa589d19c1f43c7a2f1572b958619b7"
format_stamp: "Formatted at 2026-08-07 15:15:29 on dist-test-slave-42z9"
server_key: "758c52e85d67406632dea9d39beaef1c"
server_key_iv: "4fa9e8a0e2db9e6fa44b3e98e1c1d1c0"
server_key_version: "encryptionkey@0"
I20260807 15:15:29.358923 24024 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115480851507-30472-0/minicluster-data/master-0/wal/instance:
uuid: "caa589d19c1f43c7a2f1572b958619b7"
format_stamp: "Formatted at 2026-08-07 15:15:29 on dist-test-slave-42z9"
server_key: "758c52e85d67406632dea9d39beaef1c"
server_key_iv: "4fa9e8a0e2db9e6fa44b3e98e1c1d1c0"
server_key_version: "encryptionkey@0"
I20260807 15:15:29.360258 24024 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 15:15:29.361040 24046 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:15:29.361241 24024 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:15:29.361333 24024 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115480851507-30472-0/minicluster-data/master-0/data,/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115480851507-30472-0/minicluster-data/master-0/wal
uuid: "caa589d19c1f43c7a2f1572b958619b7"
format_stamp: "Formatted at 2026-08-07 15:15:29 on dist-test-slave-42z9"
server_key: "758c52e85d67406632dea9d39beaef1c"
server_key_iv: "4fa9e8a0e2db9e6fa44b3e98e1c1d1c0"
server_key_version: "encryptionkey@0"
I20260807 15:15:29.361402 24024 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115480851507-30472-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115480851507-30472-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115480851507-30472-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:15:29.375073 24024 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:15:29.375414 24024 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:15:29.375558 24024 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:15:29.379733 24024 rpc_server.cc:307] RPC server started. Bound to: 127.29.194.62:34763
I20260807 15:15:29.379798 24126 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.194.62:34763 every 8 connection(s)
I20260807 15:15:29.380144 24024 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115480851507-30472-0/minicluster-data/master-0/data/info.pb
I20260807 15:15:29.381143 24127 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:15:29.383625 24127 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P caa589d19c1f43c7a2f1572b958619b7: Bootstrap starting.
I20260807 15:15:29.384279 24127 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P caa589d19c1f43c7a2f1572b958619b7: Neither blocks nor log segments found. Creating new log.
I20260807 15:15:29.384574 24127 log.cc:824] T 00000000000000000000000000000000 P caa589d19c1f43c7a2f1572b958619b7: Log is configured to *not* fsync() on all Append() calls
I20260807 15:15:29.385326 24127 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P caa589d19c1f43c7a2f1572b958619b7: No bootstrap required, opened a new log
I20260807 15:15:29.386186 30472 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7fZSIL/build/release/bin/kudu as pid 24024
I20260807 15:15:29.386432 30472 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115480851507-30472-0/minicluster-data/master-0/wal/instance
I20260807 15:15:29.386834 30472 external_mini_cluster.cc:1428] Setting key 5fa678c2774d6a4c18f483f9b1c0c536
I20260807 15:15:29.388298 24127 raft_consensus.cc:348] T 00000000000000000000000000000000 P caa589d19c1f43c7a2f1572b958619b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "caa589d19c1f43c7a2f1572b958619b7" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 34763 } }
I20260807 15:15:29.388484 24127 raft_consensus.cc:374] T 00000000000000000000000000000000 P caa589d19c1f43c7a2f1572b958619b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:15:29.388509 24127 raft_consensus.cc:729] T 00000000000000000000000000000000 P caa589d19c1f43c7a2f1572b958619b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: caa589d19c1f43c7a2f1572b958619b7, State: Initialized, Role: FOLLOWER
I20260807 15:15:29.388672 24127 consensus_queue.cc:260] T 00000000000000000000000000000000 P caa589d19c1f43c7a2f1572b958619b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "caa589d19c1f43c7a2f1572b958619b7" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 34763 } }
I20260807 15:15:29.388777 24127 raft_consensus.cc:388] T 00000000000000000000000000000000 P caa589d19c1f43c7a2f1572b958619b7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:15:29.388810 24127 raft_consensus.cc:482] T 00000000000000000000000000000000 P caa589d19c1f43c7a2f1572b958619b7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:15:29.388865 24127 raft_consensus.cc:3037] T 00000000000000000000000000000000 P caa589d19c1f43c7a2f1572b958619b7 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:15:29.389557 24127 raft_consensus.cc:504] T 00000000000000000000000000000000 P caa589d19c1f43c7a2f1572b958619b7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "caa589d19c1f43c7a2f1572b958619b7" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 34763 } }
I20260807 15:15:29.389659 24127 leader_election.cc:302] T 00000000000000000000000000000000 P caa589d19c1f43c7a2f1572b958619b7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: caa589d19c1f43c7a2f1572b958619b7; no voters:
I20260807 15:15:29.389848 24127 leader_election.cc:288] T 00000000000000000000000000000000 P caa589d19c1f43c7a2f1572b958619b7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 15:15:29.389891 24137 raft_consensus.cc:2781] T 00000000000000000000000000000000 P caa589d19c1f43c7a2f1572b958619b7 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:15:29.390013 24137 raft_consensus.cc:686] T 00000000000000000000000000000000 P caa589d19c1f43c7a2f1572b958619b7 [term 1 LEADER]: Becoming Leader. State: Replica: caa589d19c1f43c7a2f1572b958619b7, State: Running, Role: LEADER
I20260807 15:15:29.390100 24137 consensus_queue.cc:237] T 00000000000000000000000000000000 P caa589d19c1f43c7a2f1572b958619b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "caa589d19c1f43c7a2f1572b958619b7" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 34763 } }
I20260807 15:15:29.390141 24127 sys_catalog.cc:564] T 00000000000000000000000000000000 P caa589d19c1f43c7a2f1572b958619b7 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:15:29.390496 24139 sys_catalog.cc:455] T 00000000000000000000000000000000 P caa589d19c1f43c7a2f1572b958619b7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "caa589d19c1f43c7a2f1572b958619b7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "caa589d19c1f43c7a2f1572b958619b7" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 34763 } } }
I20260807 15:15:29.390586 24139 sys_catalog.cc:458] T 00000000000000000000000000000000 P caa589d19c1f43c7a2f1572b958619b7 [sys.catalog]: This master's current role is: LEADER
I20260807 15:15:29.390836 24140 sys_catalog.cc:455] T 00000000000000000000000000000000 P caa589d19c1f43c7a2f1572b958619b7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader caa589d19c1f43c7a2f1572b958619b7. Latest consensus state: current_term: 1 leader_uuid: "caa589d19c1f43c7a2f1572b958619b7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "caa589d19c1f43c7a2f1572b958619b7" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 34763 } } }
I20260807 15:15:29.391366 24140 sys_catalog.cc:458] T 00000000000000000000000000000000 P caa589d19c1f43c7a2f1572b958619b7 [sys.catalog]: This master's current role is: LEADER
I20260807 15:15:29.391557 24144 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:15:29.392262 24144 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:15:29.394054 24136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:29.387671 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42738 (local address 127.29.194.62:34763)
0807 15:15:29.387949 (+ 278us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:29.387958 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:29.388001 (+ 43us) server_negotiation.cc:407] Connection header received
0807 15:15:29.388285 (+ 284us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:29.388297 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:29.388395 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:29.388591 (+ 196us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:15:29.388854 (+ 263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:29.390725 (+ 1871us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:29.391899 (+ 1174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:29.392332 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:29.392496 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:29.393230 (+ 734us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:15:29.393233 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:15:29.393234 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:15:29.393472 (+ 238us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:15:29.393521 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:15:29.393538 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:15:29.393611 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0807 15:15:29.393723 (+ 112us) server_negotiation.cc:299] Negotiation successful
0807 15:15:29.393818 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":182,"thread_start_us":111,"threads_started":1}
I20260807 15:15:29.394170 24144 catalog_manager.cc:1347] Generated new cluster ID: 1f1f03398436447bbff54b3ea449dee4
I20260807 15:15:29.394217 24144 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:15:29.402293 24144 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:15:29.403065 24144 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:15:29.407931 24144 catalog_manager.cc:5868] T 00000000000000000000000000000000 P caa589d19c1f43c7a2f1572b958619b7: Generated new TSK 0
I20260807 15:15:29.408180 24144 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 15:15:29.409785 30472 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7fZSIL/build/release/bin/kudu
/tmp/dist-test-task7fZSIL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115480851507-30472-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115480851507-30472-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115480851507-30472-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115480851507-30472-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.194.1:0
--local_ip_for_outbound_sockets=127.29.194.1
--webserver_interface=127.29.194.1
--webserver_port=0
--tserver_master_addrs=127.29.194.62:34763
--builtin_ntp_servers=127.29.194.20:33799
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260807 15:15:29.495757 24166 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:15:29.496042 24166 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:15:29.496078 24166 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:15:29.497290 24166 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:15:29.497337 24166 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:15:29.497375 24166 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.194.1
I20260807 15:15:29.498863 24166 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.194.20:33799
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115480851507-30472-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115480851507-30472-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.194.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115480851507-30472-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.194.1
--webserver_port=0
--tserver_master_addrs=127.29.194.62:34763
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24166
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.194.1
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115480851507-30472-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:15:29.499132 24166 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:15:29.499456 24166 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:15:29.504706 24173 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:15:29.504680 24174 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:15:29.504871 24166 server_base.cc:1034] running on GCE node
W20260807 15:15:29.504670 24176 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:15:29.505244 24166 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:15:29.505551 24166 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:15:29.506729 24166 hybrid_clock.cc:648] HybridClock initialized: now 1786115729506712 us; error 32 us; skew 500 ppm
I20260807 15:15:29.508087 24166 webserver.cc:460] Webserver started at http://127.29.194.1:40391/ using document root <none> and password file <none>
I20260807 15:15:29.508330 24166 fs_manager.cc:359] Metadata directory not provided
I20260807 15:15:29.508380 24166 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:15:29.508517 24166 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:15:29.509671 24166 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115480851507-30472-0/minicluster-data/ts-0/data/instance:
uuid: "499d5bda369e428b9959486c40b077b3"
format_stamp: "Formatted at 2026-08-07 15:15:29 on dist-test-slave-42z9"
server_key: "8b048b25a1838829b5906b92333e9777"
server_key_iv: "bd64276f52c0fdcb7193877119796950"
server_key_version: "encryptionkey@0"
I20260807 15:15:29.510020 24166 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115480851507-30472-0/minicluster-data/ts-0/wal/instance:
uuid: "499d5bda369e428b9959486c40b077b3"
format_stamp: "Formatted at 2026-08-07 15:15:29 on dist-test-slave-42z9"
server_key: "8b048b25a1838829b5906b92333e9777"
server_key_iv: "bd64276f52c0fdcb7193877119796950"
server_key_version: "encryptionkey@0"
I20260807 15:15:29.511332 24166 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 15:15:29.512095 24186 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:15:29.512279 24166 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:15:29.512355 24166 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115480851507-30472-0/minicluster-data/ts-0/data,/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115480851507-30472-0/minicluster-data/ts-0/wal
uuid: "499d5bda369e428b9959486c40b077b3"
format_stamp: "Formatted at 2026-08-07 15:15:29 on dist-test-slave-42z9"
server_key: "8b048b25a1838829b5906b92333e9777"
server_key_iv: "bd64276f52c0fdcb7193877119796950"
server_key_version: "encryptionkey@0"
I20260807 15:15:29.512431 24166 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115480851507-30472-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115480851507-30472-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115480851507-30472-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:15:29.538395 24166 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:15:29.538745 24166 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:15:29.538866 24166 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:15:29.539113 24166 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:15:29.539475 24166 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:15:29.539510 24166 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:15:29.539541 24166 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:15:29.539562 24166 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:15:29.545562 24166 rpc_server.cc:307] RPC server started. Bound to: 127.29.194.1:44595
I20260807 15:15:29.545617 24384 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.194.1:44595 every 8 connection(s)
I20260807 15:15:29.545972 24166 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115480851507-30472-0/minicluster-data/ts-0/data/info.pb
I20260807 15:15:29.551982 24136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:29.546740 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.194.1:37993 (local address 127.29.194.62:34763)
0807 15:15:29.546805 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:29.546807 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:29.547053 (+ 246us) server_negotiation.cc:407] Connection header received
0807 15:15:29.547547 (+ 494us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:29.547549 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:29.547561 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:29.547630 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:15:29.548404 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:29.549148 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:29.550489 (+ 1341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:29.550717 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:29.550783 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:29.551644 (+ 861us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:15:29.551646 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:15:29.551646 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:15:29.551655 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:15:29.551671 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:15:29.551676 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:15:29.551739 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0807 15:15:29.551873 (+ 134us) server_negotiation.cc:299] Negotiation successful
0807 15:15:29.551912 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260807 15:15:29.552131 24389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:29.546730 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:34763 (local address 127.29.194.1:37993)
0807 15:15:29.546923 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:29.546974 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0807 15:15:29.547455 (+ 481us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:15:29.547701 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:15:29.547704 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:15:29.547713 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:15:29.548244 (+ 531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:29.548248 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:29.549303 (+ 1055us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:29.549304 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:29.550368 (+ 1064us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:29.550372 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:29.550791 (+ 419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:29.550792 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:29.550915 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:29.551489 (+ 574us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 15:15:29.551489 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:15:29.551521 (+ 32us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 15:15:29.551548 (+ 27us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:15:29.551764 (+ 216us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:15:29.551773 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:15:29.551777 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 15:15:29.551834 (+ 57us) client_negotiation.cc:240] Negotiation successful
0807 15:15:29.551927 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":79,"threads_started":1}
I20260807 15:15:29.552605 24386 heartbeater.cc:347] Connected to a master server at 127.29.194.62:34763
I20260807 15:15:29.552707 24386 heartbeater.cc:464] Registering TS with master...
I20260807 15:15:29.552891 24386 heartbeater.cc:511] Master 127.29.194.62:34763 requested a full tablet report, sending...
I20260807 15:15:29.553593 24071 ts_manager.cc:194] Registered new tserver with Master: 499d5bda369e428b9959486c40b077b3 (127.29.194.1:44595)
I20260807 15:15:29.554220 24071 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.29.194.1:37993
I20260807 15:15:29.556236 30472 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7fZSIL/build/release/bin/kudu as pid 24166
I20260807 15:15:29.556411 30472 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115480851507-30472-0/minicluster-data/ts-0/wal/instance
I20260807 15:15:29.556672 30472 external_mini_cluster.cc:1428] Setting key a12ea10f8ba9a2039fba41b81914bd5d
I20260807 15:15:29.557694 30472 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7fZSIL/build/release/bin/kudu
/tmp/dist-test-task7fZSIL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115480851507-30472-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115480851507-30472-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115480851507-30472-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115480851507-30472-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.194.2:0
--local_ip_for_outbound_sockets=127.29.194.2
--webserver_interface=127.29.194.2
--webserver_port=0
--tserver_master_addrs=127.29.194.62:34763
--builtin_ntp_servers=127.29.194.20:33799
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260807 15:15:29.644932 24391 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:15:29.645177 24391 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:15:29.645215 24391 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:15:29.646955 24391 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:15:29.647081 24391 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:15:29.647145 24391 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.194.2
I20260807 15:15:29.649629 24391 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.194.20:33799
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115480851507-30472-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115480851507-30472-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.194.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115480851507-30472-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.194.2
--webserver_port=0
--tserver_master_addrs=127.29.194.62:34763
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24391
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.194.2
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115480851507-30472-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:15:29.650063 24391 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:15:29.650488 24391 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:15:29.656306 24400 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:15:29.656293 24401 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:15:29.656330 24391 server_base.cc:1034] running on GCE node
W20260807 15:15:29.656270 24403 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:15:29.656785 24391 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:15:29.657051 24391 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:15:29.658231 24391 hybrid_clock.cc:648] HybridClock initialized: now 1786115729658213 us; error 32 us; skew 500 ppm
I20260807 15:15:29.659607 24391 webserver.cc:460] Webserver started at http://127.29.194.2:45903/ using document root <none> and password file <none>
I20260807 15:15:29.659852 24391 fs_manager.cc:359] Metadata directory not provided
I20260807 15:15:29.659907 24391 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:15:29.660053 24391 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:15:29.661151 24391 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115480851507-30472-0/minicluster-data/ts-1/data/instance:
uuid: "0c1c121d979444428857f8aa2612e077"
format_stamp: "Formatted at 2026-08-07 15:15:29 on dist-test-slave-42z9"
server_key: "c929b46572ed1be51cd8296dc92271e5"
server_key_iv: "156e562aa803ac1f1beb34dea00d705a"
server_key_version: "encryptionkey@0"
I20260807 15:15:29.661553 24391 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115480851507-30472-0/minicluster-data/ts-1/wal/instance:
uuid: "0c1c121d979444428857f8aa2612e077"
format_stamp: "Formatted at 2026-08-07 15:15:29 on dist-test-slave-42z9"
server_key: "c929b46572ed1be51cd8296dc92271e5"
server_key_iv: "156e562aa803ac1f1beb34dea00d705a"
server_key_version: "encryptionkey@0"
I20260807 15:15:29.663105 24391 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:15:29.664336 24411 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:15:29.664556 24391 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:15:29.664657 24391 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115480851507-30472-0/minicluster-data/ts-1/data,/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115480851507-30472-0/minicluster-data/ts-1/wal
uuid: "0c1c121d979444428857f8aa2612e077"
format_stamp: "Formatted at 2026-08-07 15:15:29 on dist-test-slave-42z9"
server_key: "c929b46572ed1be51cd8296dc92271e5"
server_key_iv: "156e562aa803ac1f1beb34dea00d705a"
server_key_version: "encryptionkey@0"
I20260807 15:15:29.664747 24391 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115480851507-30472-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115480851507-30472-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115480851507-30472-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:15:29.677803 24391 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:15:29.678150 24391 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:15:29.678274 24391 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:15:29.678529 24391 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:15:29.678875 24391 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:15:29.678908 24391 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:15:29.678930 24391 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:15:29.678954 24391 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:15:29.684891 24391 rpc_server.cc:307] RPC server started. Bound to: 127.29.194.2:38241
I20260807 15:15:29.684935 24588 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.194.2:38241 every 8 connection(s)
I20260807 15:15:29.685261 24391 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115480851507-30472-0/minicluster-data/ts-1/data/info.pb
I20260807 15:15:29.691112 24136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:29.686507 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.194.2:34523 (local address 127.29.194.62:34763)
0807 15:15:29.686599 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:29.686601 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:29.686775 (+ 174us) server_negotiation.cc:407] Connection header received
0807 15:15:29.687256 (+ 481us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:29.687258 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:29.687268 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:29.687324 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:15:29.688098 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:29.688633 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:29.689750 (+ 1117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:29.689943 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:29.690004 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:29.690830 (+ 826us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:15:29.690832 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:15:29.690832 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:15:29.690837 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:15:29.690860 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:15:29.690864 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:15:29.690941 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0807 15:15:29.691024 (+ 83us) server_negotiation.cc:299] Negotiation successful
0807 15:15:29.691049 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260807 15:15:29.691339 24593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:29.686388 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:34763 (local address 127.29.194.2:34523)
0807 15:15:29.686612 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:29.686661 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0807 15:15:29.687163 (+ 502us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:15:29.687459 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:15:29.687464 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:15:29.687474 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:15:29.687998 (+ 524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:29.688002 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:29.688736 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:29.688737 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:29.689630 (+ 893us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:29.689633 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:29.690036 (+ 403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:29.690037 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:29.690154 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:29.690689 (+ 535us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 15:15:29.690689 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:15:29.690721 (+ 32us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 15:15:29.690753 (+ 32us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:15:29.690945 (+ 192us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:15:29.690951 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:15:29.690954 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 15:15:29.691021 (+ 67us) client_negotiation.cc:240] Negotiation successful
0807 15:15:29.691108 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":80,"threads_started":1}
I20260807 15:15:29.691896 24589 heartbeater.cc:347] Connected to a master server at 127.29.194.62:34763
I20260807 15:15:29.692022 24589 heartbeater.cc:464] Registering TS with master...
I20260807 15:15:29.692202 24589 heartbeater.cc:511] Master 127.29.194.62:34763 requested a full tablet report, sending...
I20260807 15:15:29.692646 24071 ts_manager.cc:194] Registered new tserver with Master: 0c1c121d979444428857f8aa2612e077 (127.29.194.2:38241)
I20260807 15:15:29.693048 24071 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.29.194.2:34523
I20260807 15:15:29.694258 30472 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7fZSIL/build/release/bin/kudu as pid 24391
I20260807 15:15:29.694343 30472 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115480851507-30472-0/minicluster-data/ts-1/wal/instance
I20260807 15:15:29.694531 30472 external_mini_cluster.cc:1428] Setting key e3039e4f58c731cf36f20347e3085bcf
I20260807 15:15:29.695334 30472 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7fZSIL/build/release/bin/kudu
/tmp/dist-test-task7fZSIL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115480851507-30472-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115480851507-30472-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115480851507-30472-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115480851507-30472-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.194.3:0
--local_ip_for_outbound_sockets=127.29.194.3
--webserver_interface=127.29.194.3
--webserver_port=0
--tserver_master_addrs=127.29.194.62:34763
--builtin_ntp_servers=127.29.194.20:33799
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260807 15:15:29.778618 24594 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:15:29.778836 24594 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:15:29.778867 24594 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:15:29.780053 24594 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:15:29.780099 24594 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:15:29.780133 24594 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.194.3
I20260807 15:15:29.781541 24594 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.194.20:33799
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115480851507-30472-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115480851507-30472-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.194.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115480851507-30472-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.194.3
--webserver_port=0
--tserver_master_addrs=127.29.194.62:34763
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24594
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.194.3
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115480851507-30472-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:15:29.781813 24594 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:15:29.782111 24594 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:15:29.787407 24611 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:15:29.787626 24594 server_base.cc:1034] running on GCE node
W20260807 15:15:29.787408 24609 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:15:29.787436 24613 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:15:29.788089 24594 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:15:29.788489 24594 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:15:29.789651 24594 hybrid_clock.cc:648] HybridClock initialized: now 1786115729789637 us; error 37 us; skew 500 ppm
I20260807 15:15:29.791196 24594 webserver.cc:460] Webserver started at http://127.29.194.3:34961/ using document root <none> and password file <none>
I20260807 15:15:29.791453 24594 fs_manager.cc:359] Metadata directory not provided
I20260807 15:15:29.791501 24594 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:15:29.791643 24594 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:15:29.792768 24594 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115480851507-30472-0/minicluster-data/ts-2/data/instance:
uuid: "39f290ad1d524320b20d18041dc24e7a"
format_stamp: "Formatted at 2026-08-07 15:15:29 on dist-test-slave-42z9"
server_key: "a9ebc3b14e25c7e50b5a90c0e92ec9ff"
server_key_iv: "631d8bbe61c01bbe1017fe277d65d784"
server_key_version: "encryptionkey@0"
I20260807 15:15:29.793113 24594 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115480851507-30472-0/minicluster-data/ts-2/wal/instance:
uuid: "39f290ad1d524320b20d18041dc24e7a"
format_stamp: "Formatted at 2026-08-07 15:15:29 on dist-test-slave-42z9"
server_key: "a9ebc3b14e25c7e50b5a90c0e92ec9ff"
server_key_iv: "631d8bbe61c01bbe1017fe277d65d784"
server_key_version: "encryptionkey@0"
I20260807 15:15:29.794590 24594 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260807 15:15:29.795555 24622 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:15:29.795802 24594 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 15:15:29.795897 24594 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115480851507-30472-0/minicluster-data/ts-2/data,/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115480851507-30472-0/minicluster-data/ts-2/wal
uuid: "39f290ad1d524320b20d18041dc24e7a"
format_stamp: "Formatted at 2026-08-07 15:15:29 on dist-test-slave-42z9"
server_key: "a9ebc3b14e25c7e50b5a90c0e92ec9ff"
server_key_iv: "631d8bbe61c01bbe1017fe277d65d784"
server_key_version: "encryptionkey@0"
I20260807 15:15:29.795969 24594 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115480851507-30472-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115480851507-30472-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115480851507-30472-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:15:29.810715 24594 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:15:29.811053 24594 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:15:29.811196 24594 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:15:29.811451 24594 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:15:29.811795 24594 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:15:29.811832 24594 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:15:29.811864 24594 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:15:29.811882 24594 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:15:29.817637 24594 rpc_server.cc:307] RPC server started. Bound to: 127.29.194.3:38471
I20260807 15:15:29.817679 24810 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.194.3:38471 every 8 connection(s)
I20260807 15:15:29.818029 24594 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115480851507-30472-0/minicluster-data/ts-2/data/info.pb
I20260807 15:15:29.821902 30472 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7fZSIL/build/release/bin/kudu as pid 24594
I20260807 15:15:29.822031 30472 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115480851507-30472-0/minicluster-data/ts-2/wal/instance
I20260807 15:15:29.822219 30472 external_mini_cluster.cc:1428] Setting key 83c1e99b640fedcf2170baeac304e3d5
I20260807 15:15:29.824096 24136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:29.818792 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.194.3:51309 (local address 127.29.194.62:34763)
0807 15:15:29.818875 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:29.818877 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:29.819093 (+ 216us) server_negotiation.cc:407] Connection header received
0807 15:15:29.819613 (+ 520us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:29.819614 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:29.819624 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:29.819683 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:15:29.820456 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:29.821094 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:29.822537 (+ 1443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:29.822829 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:29.822902 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:29.823866 (+ 964us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:15:29.823868 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:15:29.823868 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:15:29.823873 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:15:29.823889 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:15:29.823893 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:15:29.823954 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0807 15:15:29.824002 (+ 48us) server_negotiation.cc:299] Negotiation successful
0807 15:15:29.824035 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260807 15:15:29.824319 24815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:29.818763 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:34763 (local address 127.29.194.3:51309)
0807 15:15:29.818974 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:29.819009 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0807 15:15:29.819501 (+ 492us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:15:29.819745 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:15:29.819749 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:15:29.819758 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:15:29.820282 (+ 524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:29.820287 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:29.821264 (+ 977us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:29.821265 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:29.822459 (+ 1194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:29.822462 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:29.822930 (+ 468us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:29.822931 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:29.823117 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:29.823706 (+ 589us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 15:15:29.823706 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:15:29.823738 (+ 32us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 15:15:29.823763 (+ 25us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:15:29.823949 (+ 186us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:15:29.823952 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:15:29.823955 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 15:15:29.824013 (+ 58us) client_negotiation.cc:240] Negotiation successful
0807 15:15:29.824104 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":88,"threads_started":1}
I20260807 15:15:29.824815 24812 heartbeater.cc:347] Connected to a master server at 127.29.194.62:34763
I20260807 15:15:29.824910 24812 heartbeater.cc:464] Registering TS with master...
I20260807 15:15:29.825080 24812 heartbeater.cc:511] Master 127.29.194.62:34763 requested a full tablet report, sending...
I20260807 15:15:29.825493 24071 ts_manager.cc:194] Registered new tserver with Master: 39f290ad1d524320b20d18041dc24e7a (127.29.194.3:38471)
I20260807 15:15:29.825845 24071 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.29.194.3:51309
I20260807 15:15:29.834282 30472 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 15:15:29.838562 24136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:29.835666 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48102 (local address 127.29.194.62:34763)
0807 15:15:29.835739 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:29.835741 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:29.835867 (+ 126us) server_negotiation.cc:407] Connection header received
0807 15:15:29.835893 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:29.835894 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:29.835907 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:29.835948 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:15:29.836080 (+ 132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:29.836596 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:29.837302 (+ 706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:29.837535 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:29.837587 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:29.838225 (+ 638us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:15:29.838226 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:15:29.838227 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:15:29.838231 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:15:29.838246 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:15:29.838249 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:15:29.838306 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0807 15:15:29.838445 (+ 139us) server_negotiation.cc:299] Negotiation successful
0807 15:15:29.838469 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260807 15:15:29.840180 24071 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48102:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 15:15:29.842020 24071 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 15:15:29.849792 24826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:29.844403 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.3:38471 (local address 127.0.0.1:51118)
0807 15:15:29.844800 (+ 397us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:29.844810 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 15:15:29.844892 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:15:29.845121 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:15:29.845122 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:15:29.845124 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:15:29.845179 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:29.845180 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:29.846536 (+ 1356us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:29.846538 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:29.848359 (+ 1821us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:29.848364 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:29.849396 (+ 1032us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:29.849398 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:29.849545 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:29.849564 (+ 19us) client_negotiation.cc:769] Sending connection context
0807 15:15:29.849637 (+ 73us) client_negotiation.cc:240] Negotiation successful
0807 15:15:29.849700 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":379,"mutex_wait_us":58,"thread_start_us":120,"threads_started":1}
I20260807 15:15:29.849835 24828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:29.844551 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51118 (local address 127.29.194.3:38471)
0807 15:15:29.844742 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:29.844746 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:29.844868 (+ 122us) server_negotiation.cc:407] Connection header received
0807 15:15:29.844978 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:29.844981 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:29.844997 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:29.845041 (+ 44us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:15:29.845255 (+ 214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:29.846444 (+ 1189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:29.848504 (+ 2060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:29.849296 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:29.849373 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:29.849508 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0807 15:15:29.849694 (+ 186us) server_negotiation.cc:299] Negotiation successful
0807 15:15:29.849746 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":156,"thread_start_us":83,"threads_started":1}
I20260807 15:15:29.850639 24825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:29.844612 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.2:38241 (local address 127.0.0.1:49172)
0807 15:15:29.844800 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:29.844810 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 15:15:29.844877 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:15:29.845092 (+ 215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:15:29.845095 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:15:29.845097 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:15:29.845178 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:29.845180 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:29.847768 (+ 2588us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:29.847771 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:29.848829 (+ 1058us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:29.848834 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:29.849817 (+ 983us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:29.849818 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:29.850435 (+ 617us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:29.850443 (+ 8us) client_negotiation.cc:769] Sending connection context
0807 15:15:29.850499 (+ 56us) client_negotiation.cc:240] Negotiation successful
0807 15:15:29.850556 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":175,"mutex_wait_us":3,"spinlock_wait_cycles":384,"thread_start_us":76,"threads_started":1}
I20260807 15:15:29.850747 24829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:29.844557 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56506 (local address 127.29.194.1:44595)
0807 15:15:29.844742 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:29.844746 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:29.844974 (+ 228us) server_negotiation.cc:407] Connection header received
0807 15:15:29.845018 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:29.845021 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:29.845036 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:29.845099 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:15:29.845283 (+ 184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:29.846444 (+ 1161us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:29.849551 (+ 3107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:29.850243 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:29.850306 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:29.850425 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0807 15:15:29.850618 (+ 193us) server_negotiation.cc:299] Negotiation successful
0807 15:15:29.850659 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":149,"thread_start_us":77,"threads_started":1}
I20260807 15:15:29.850714 24694 tablet_service.cc:1467] Processing CreateTablet for tablet 46e5ff15c6854882b54fc91fd49b2f8c (DEFAULT_TABLE table=test-table [id=c3908f06057b486c9b99336d00febf37]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:15:29.851084 24832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:29.844734 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49172 (local address 127.29.194.2:38241)
0807 15:15:29.844927 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:29.844932 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:29.844943 (+ 11us) server_negotiation.cc:407] Connection header received
0807 15:15:29.844984 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:29.844986 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:29.845001 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:29.845082 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:15:29.846562 (+ 1480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:29.847496 (+ 934us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:29.848981 (+ 1485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:29.849623 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:29.849695 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:29.849777 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0807 15:15:29.850940 (+ 1163us) server_negotiation.cc:299] Negotiation successful
0807 15:15:29.850994 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":159,"thread_start_us":88,"threads_started":1}
I20260807 15:15:29.851146 24694 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 46e5ff15c6854882b54fc91fd49b2f8c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:15:29.850744 24830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:29.844390 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.1:44595 (local address 127.0.0.1:56506)
0807 15:15:29.844754 (+ 364us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:29.844799 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0807 15:15:29.844872 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:15:29.845164 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:15:29.845165 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:15:29.845167 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:15:29.845208 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:29.845210 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:29.846536 (+ 1326us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:29.846538 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:29.848358 (+ 1820us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:29.848364 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:29.850463 (+ 2099us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:29.850465 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:29.850535 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:29.850539 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 15:15:29.850590 (+ 51us) client_negotiation.cc:240] Negotiation successful
0807 15:15:29.850637 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":333,"mutex_wait_us":95,"thread_start_us":64,"threads_started":1}
I20260807 15:15:29.852108 24486 tablet_service.cc:1467] Processing CreateTablet for tablet 46e5ff15c6854882b54fc91fd49b2f8c (DEFAULT_TABLE table=test-table [id=c3908f06057b486c9b99336d00febf37]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:15:29.852494 24486 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 46e5ff15c6854882b54fc91fd49b2f8c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:15:29.854362 24265 tablet_service.cc:1467] Processing CreateTablet for tablet 46e5ff15c6854882b54fc91fd49b2f8c (DEFAULT_TABLE table=test-table [id=c3908f06057b486c9b99336d00febf37]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:15:29.854789 24833 tablet_bootstrap.cc:492] T 46e5ff15c6854882b54fc91fd49b2f8c P 39f290ad1d524320b20d18041dc24e7a: Bootstrap starting.
I20260807 15:15:29.854879 24265 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 46e5ff15c6854882b54fc91fd49b2f8c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:15:29.855666 24833 tablet_bootstrap.cc:654] T 46e5ff15c6854882b54fc91fd49b2f8c P 39f290ad1d524320b20d18041dc24e7a: Neither blocks nor log segments found. Creating new log.
I20260807 15:15:29.855844 24834 tablet_bootstrap.cc:492] T 46e5ff15c6854882b54fc91fd49b2f8c P 0c1c121d979444428857f8aa2612e077: Bootstrap starting.
I20260807 15:15:29.856027 24833 log.cc:824] T 46e5ff15c6854882b54fc91fd49b2f8c P 39f290ad1d524320b20d18041dc24e7a: Log is configured to *not* fsync() on all Append() calls
I20260807 15:15:29.856467 24834 tablet_bootstrap.cc:654] T 46e5ff15c6854882b54fc91fd49b2f8c P 0c1c121d979444428857f8aa2612e077: Neither blocks nor log segments found. Creating new log.
I20260807 15:15:29.856793 24834 log.cc:824] T 46e5ff15c6854882b54fc91fd49b2f8c P 0c1c121d979444428857f8aa2612e077: Log is configured to *not* fsync() on all Append() calls
I20260807 15:15:29.857259 24833 tablet_bootstrap.cc:492] T 46e5ff15c6854882b54fc91fd49b2f8c P 39f290ad1d524320b20d18041dc24e7a: No bootstrap required, opened a new log
I20260807 15:15:29.857331 24833 ts_tablet_manager.cc:1397] T 46e5ff15c6854882b54fc91fd49b2f8c P 39f290ad1d524320b20d18041dc24e7a: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260807 15:15:29.857827 24834 tablet_bootstrap.cc:492] T 46e5ff15c6854882b54fc91fd49b2f8c P 0c1c121d979444428857f8aa2612e077: No bootstrap required, opened a new log
I20260807 15:15:29.857904 24834 ts_tablet_manager.cc:1397] T 46e5ff15c6854882b54fc91fd49b2f8c P 0c1c121d979444428857f8aa2612e077: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 15:15:29.858865 24838 tablet_bootstrap.cc:492] T 46e5ff15c6854882b54fc91fd49b2f8c P 499d5bda369e428b9959486c40b077b3: Bootstrap starting.
I20260807 15:15:29.859525 24838 tablet_bootstrap.cc:654] T 46e5ff15c6854882b54fc91fd49b2f8c P 499d5bda369e428b9959486c40b077b3: Neither blocks nor log segments found. Creating new log.
I20260807 15:15:29.859797 24838 log.cc:824] T 46e5ff15c6854882b54fc91fd49b2f8c P 499d5bda369e428b9959486c40b077b3: Log is configured to *not* fsync() on all Append() calls
I20260807 15:15:29.859977 24833 raft_consensus.cc:348] T 46e5ff15c6854882b54fc91fd49b2f8c P 39f290ad1d524320b20d18041dc24e7a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "499d5bda369e428b9959486c40b077b3" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 44595 } } peers { permanent_uuid: "39f290ad1d524320b20d18041dc24e7a" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 38471 } } peers { permanent_uuid: "0c1c121d979444428857f8aa2612e077" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 38241 } }
I20260807 15:15:29.860180 24833 raft_consensus.cc:374] T 46e5ff15c6854882b54fc91fd49b2f8c P 39f290ad1d524320b20d18041dc24e7a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:15:29.860215 24833 raft_consensus.cc:729] T 46e5ff15c6854882b54fc91fd49b2f8c P 39f290ad1d524320b20d18041dc24e7a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 39f290ad1d524320b20d18041dc24e7a, State: Initialized, Role: FOLLOWER
I20260807 15:15:29.860284 24834 raft_consensus.cc:348] T 46e5ff15c6854882b54fc91fd49b2f8c P 0c1c121d979444428857f8aa2612e077 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "499d5bda369e428b9959486c40b077b3" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 44595 } } peers { permanent_uuid: "39f290ad1d524320b20d18041dc24e7a" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 38471 } } peers { permanent_uuid: "0c1c121d979444428857f8aa2612e077" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 38241 } }
I20260807 15:15:29.860369 24833 consensus_queue.cc:260] T 46e5ff15c6854882b54fc91fd49b2f8c P 39f290ad1d524320b20d18041dc24e7a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "499d5bda369e428b9959486c40b077b3" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 44595 } } peers { permanent_uuid: "39f290ad1d524320b20d18041dc24e7a" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 38471 } } peers { permanent_uuid: "0c1c121d979444428857f8aa2612e077" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 38241 } }
I20260807 15:15:29.860430 24834 raft_consensus.cc:374] T 46e5ff15c6854882b54fc91fd49b2f8c P 0c1c121d979444428857f8aa2612e077 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:15:29.860473 24834 raft_consensus.cc:729] T 46e5ff15c6854882b54fc91fd49b2f8c P 0c1c121d979444428857f8aa2612e077 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0c1c121d979444428857f8aa2612e077, State: Initialized, Role: FOLLOWER
I20260807 15:15:29.860594 24834 consensus_queue.cc:260] T 46e5ff15c6854882b54fc91fd49b2f8c P 0c1c121d979444428857f8aa2612e077 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "499d5bda369e428b9959486c40b077b3" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 44595 } } peers { permanent_uuid: "39f290ad1d524320b20d18041dc24e7a" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 38471 } } peers { permanent_uuid: "0c1c121d979444428857f8aa2612e077" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 38241 } }
I20260807 15:15:29.860641 24833 ts_tablet_manager.cc:1428] T 46e5ff15c6854882b54fc91fd49b2f8c P 39f290ad1d524320b20d18041dc24e7a: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260807 15:15:29.860759 24812 heartbeater.cc:503] Master 127.29.194.62:34763 was elected leader, sending a full tablet report...
I20260807 15:15:29.860841 24834 ts_tablet_manager.cc:1428] T 46e5ff15c6854882b54fc91fd49b2f8c P 0c1c121d979444428857f8aa2612e077: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260807 15:15:29.860940 24589 heartbeater.cc:503] Master 127.29.194.62:34763 was elected leader, sending a full tablet report...
I20260807 15:15:29.861044 24838 tablet_bootstrap.cc:492] T 46e5ff15c6854882b54fc91fd49b2f8c P 499d5bda369e428b9959486c40b077b3: No bootstrap required, opened a new log
I20260807 15:15:29.861129 24838 ts_tablet_manager.cc:1397] T 46e5ff15c6854882b54fc91fd49b2f8c P 499d5bda369e428b9959486c40b077b3: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 15:15:29.863245 24838 raft_consensus.cc:348] T 46e5ff15c6854882b54fc91fd49b2f8c P 499d5bda369e428b9959486c40b077b3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "499d5bda369e428b9959486c40b077b3" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 44595 } } peers { permanent_uuid: "39f290ad1d524320b20d18041dc24e7a" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 38471 } } peers { permanent_uuid: "0c1c121d979444428857f8aa2612e077" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 38241 } }
I20260807 15:15:29.863392 24838 raft_consensus.cc:374] T 46e5ff15c6854882b54fc91fd49b2f8c P 499d5bda369e428b9959486c40b077b3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:15:29.863413 24838 raft_consensus.cc:729] T 46e5ff15c6854882b54fc91fd49b2f8c P 499d5bda369e428b9959486c40b077b3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 499d5bda369e428b9959486c40b077b3, State: Initialized, Role: FOLLOWER
I20260807 15:15:29.863523 24838 consensus_queue.cc:260] T 46e5ff15c6854882b54fc91fd49b2f8c P 499d5bda369e428b9959486c40b077b3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "499d5bda369e428b9959486c40b077b3" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 44595 } } peers { permanent_uuid: "39f290ad1d524320b20d18041dc24e7a" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 38471 } } peers { permanent_uuid: "0c1c121d979444428857f8aa2612e077" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 38241 } }
I20260807 15:15:29.863763 24838 ts_tablet_manager.cc:1428] T 46e5ff15c6854882b54fc91fd49b2f8c P 499d5bda369e428b9959486c40b077b3: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260807 15:15:29.863821 24386 heartbeater.cc:503] Master 127.29.194.62:34763 was elected leader, sending a full tablet report...
W20260807 15:15:29.935889 24591 tablet.cc:2369] T 46e5ff15c6854882b54fc91fd49b2f8c P 0c1c121d979444428857f8aa2612e077: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 15:15:29.967317 24845 raft_consensus.cc:482] T 46e5ff15c6854882b54fc91fd49b2f8c P 0c1c121d979444428857f8aa2612e077 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 15:15:29.967451 24845 raft_consensus.cc:504] T 46e5ff15c6854882b54fc91fd49b2f8c P 0c1c121d979444428857f8aa2612e077 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "499d5bda369e428b9959486c40b077b3" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 44595 } } peers { permanent_uuid: "39f290ad1d524320b20d18041dc24e7a" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 38471 } } peers { permanent_uuid: "0c1c121d979444428857f8aa2612e077" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 38241 } }
I20260807 15:15:29.967824 24845 leader_election.cc:288] T 46e5ff15c6854882b54fc91fd49b2f8c P 0c1c121d979444428857f8aa2612e077 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 499d5bda369e428b9959486c40b077b3 (127.29.194.1:44595), 39f290ad1d524320b20d18041dc24e7a (127.29.194.3:38471)
I20260807 15:15:29.970760 24829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:29.968122 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.194.2:36799 (local address 127.29.194.1:44595)
0807 15:15:29.968187 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:29.968188 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:29.968197 (+ 9us) server_negotiation.cc:407] Connection header received
0807 15:15:29.968224 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:29.968224 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:29.968234 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:29.968312 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:15:29.968479 (+ 167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:29.968955 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:29.970043 (+ 1088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:29.970417 (+ 374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:29.970483 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:29.970493 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0807 15:15:29.970651 (+ 158us) server_negotiation.cc:299] Negotiation successful
0807 15:15:29.970691 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260807 15:15:29.970767 24828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:29.968123 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.194.2:45461 (local address 127.29.194.3:38471)
0807 15:15:29.968190 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:29.968192 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:29.968446 (+ 254us) server_negotiation.cc:407] Connection header received
0807 15:15:29.968501 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:29.968502 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:29.968511 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:29.968551 (+ 40us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:15:29.968690 (+ 139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:29.969160 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:29.970115 (+ 955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:29.970454 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:29.970522 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:29.970530 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0807 15:15:29.970665 (+ 135us) server_negotiation.cc:299] Negotiation successful
0807 15:15:29.970698 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260807 15:15:29.970775 24593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:29.968018 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.1:44595 (local address 127.29.194.2:36799)
0807 15:15:29.968080 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:29.968093 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 15:15:29.968135 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:15:29.968327 (+ 192us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:15:29.968328 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:15:29.968330 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:15:29.968395 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:29.968397 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:29.969026 (+ 629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:29.969027 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:29.969930 (+ 903us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:29.969933 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:29.970498 (+ 565us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:29.970499 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:29.970579 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:29.970583 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 15:15:29.970629 (+ 46us) client_negotiation.cc:240] Negotiation successful
0807 15:15:29.970674 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43}
I20260807 15:15:29.970788 24847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:29.968022 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.3:38471 (local address 127.29.194.2:45461)
0807 15:15:29.968376 (+ 354us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:29.968386 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 15:15:29.968440 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:15:29.968562 (+ 122us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:15:29.968562 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:15:29.968563 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:15:29.968633 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:29.968635 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:29.969237 (+ 602us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:29.969238 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:29.970032 (+ 794us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:29.970034 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:29.970546 (+ 512us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:29.970547 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:29.970605 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:29.970608 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 15:15:29.970652 (+ 44us) client_negotiation.cc:240] Negotiation successful
0807 15:15:29.970691 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":332,"mutex_wait_us":159,"thread_start_us":108,"threads_started":1}
I20260807 15:15:29.971220 24293 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "46e5ff15c6854882b54fc91fd49b2f8c" candidate_uuid: "0c1c121d979444428857f8aa2612e077" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "499d5bda369e428b9959486c40b077b3" is_pre_election: true
I20260807 15:15:29.971395 24293 raft_consensus.cc:2454] T 46e5ff15c6854882b54fc91fd49b2f8c P 499d5bda369e428b9959486c40b077b3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0c1c121d979444428857f8aa2612e077 in term 0.
I20260807 15:15:29.971374 24728 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "46e5ff15c6854882b54fc91fd49b2f8c" candidate_uuid: "0c1c121d979444428857f8aa2612e077" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39f290ad1d524320b20d18041dc24e7a" is_pre_election: true
I20260807 15:15:29.971532 24728 raft_consensus.cc:2454] T 46e5ff15c6854882b54fc91fd49b2f8c P 39f290ad1d524320b20d18041dc24e7a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0c1c121d979444428857f8aa2612e077 in term 0.
I20260807 15:15:29.971675 24415 leader_election.cc:302] T 46e5ff15c6854882b54fc91fd49b2f8c P 0c1c121d979444428857f8aa2612e077 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0c1c121d979444428857f8aa2612e077, 499d5bda369e428b9959486c40b077b3; no voters:
I20260807 15:15:29.971822 24845 raft_consensus.cc:2781] T 46e5ff15c6854882b54fc91fd49b2f8c P 0c1c121d979444428857f8aa2612e077 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 15:15:29.971896 24845 raft_consensus.cc:482] T 46e5ff15c6854882b54fc91fd49b2f8c P 0c1c121d979444428857f8aa2612e077 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 15:15:29.971963 24845 raft_consensus.cc:3037] T 46e5ff15c6854882b54fc91fd49b2f8c P 0c1c121d979444428857f8aa2612e077 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:15:29.972716 24845 raft_consensus.cc:504] T 46e5ff15c6854882b54fc91fd49b2f8c P 0c1c121d979444428857f8aa2612e077 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "499d5bda369e428b9959486c40b077b3" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 44595 } } peers { permanent_uuid: "39f290ad1d524320b20d18041dc24e7a" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 38471 } } peers { permanent_uuid: "0c1c121d979444428857f8aa2612e077" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 38241 } }
I20260807 15:15:29.972923 24845 leader_election.cc:288] T 46e5ff15c6854882b54fc91fd49b2f8c P 0c1c121d979444428857f8aa2612e077 [CANDIDATE]: Term 1 election: Requested vote from peers 499d5bda369e428b9959486c40b077b3 (127.29.194.1:44595), 39f290ad1d524320b20d18041dc24e7a (127.29.194.3:38471)
I20260807 15:15:29.973011 24293 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "46e5ff15c6854882b54fc91fd49b2f8c" candidate_uuid: "0c1c121d979444428857f8aa2612e077" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "499d5bda369e428b9959486c40b077b3"
I20260807 15:15:29.973031 24728 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "46e5ff15c6854882b54fc91fd49b2f8c" candidate_uuid: "0c1c121d979444428857f8aa2612e077" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39f290ad1d524320b20d18041dc24e7a"
I20260807 15:15:29.973104 24728 raft_consensus.cc:3037] T 46e5ff15c6854882b54fc91fd49b2f8c P 39f290ad1d524320b20d18041dc24e7a [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:15:29.973100 24293 raft_consensus.cc:3037] T 46e5ff15c6854882b54fc91fd49b2f8c P 499d5bda369e428b9959486c40b077b3 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:15:29.973901 24728 raft_consensus.cc:2454] T 46e5ff15c6854882b54fc91fd49b2f8c P 39f290ad1d524320b20d18041dc24e7a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0c1c121d979444428857f8aa2612e077 in term 1.
I20260807 15:15:29.973904 24293 raft_consensus.cc:2454] T 46e5ff15c6854882b54fc91fd49b2f8c P 499d5bda369e428b9959486c40b077b3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0c1c121d979444428857f8aa2612e077 in term 1.
I20260807 15:15:29.974061 24415 leader_election.cc:302] T 46e5ff15c6854882b54fc91fd49b2f8c P 0c1c121d979444428857f8aa2612e077 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0c1c121d979444428857f8aa2612e077, 499d5bda369e428b9959486c40b077b3; no voters:
I20260807 15:15:29.974181 24845 raft_consensus.cc:2781] T 46e5ff15c6854882b54fc91fd49b2f8c P 0c1c121d979444428857f8aa2612e077 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:15:29.974361 24845 raft_consensus.cc:686] T 46e5ff15c6854882b54fc91fd49b2f8c P 0c1c121d979444428857f8aa2612e077 [term 1 LEADER]: Becoming Leader. State: Replica: 0c1c121d979444428857f8aa2612e077, State: Running, Role: LEADER
I20260807 15:15:29.974457 24845 consensus_queue.cc:237] T 46e5ff15c6854882b54fc91fd49b2f8c P 0c1c121d979444428857f8aa2612e077 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "499d5bda369e428b9959486c40b077b3" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 44595 } } peers { permanent_uuid: "39f290ad1d524320b20d18041dc24e7a" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 38471 } } peers { permanent_uuid: "0c1c121d979444428857f8aa2612e077" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 38241 } }
I20260807 15:15:29.975358 24070 catalog_manager.cc:5495] T 46e5ff15c6854882b54fc91fd49b2f8c P 0c1c121d979444428857f8aa2612e077 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0c1c121d979444428857f8aa2612e077 (127.29.194.2). New cstate: current_term: 1 leader_uuid: "0c1c121d979444428857f8aa2612e077" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "499d5bda369e428b9959486c40b077b3" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 44595 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "39f290ad1d524320b20d18041dc24e7a" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 38471 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0c1c121d979444428857f8aa2612e077" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 38241 } health_report { overall_health: HEALTHY } } }
I20260807 15:15:29.999852 24832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:29.997674 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49178 (local address 127.29.194.2:38241)
0807 15:15:29.997724 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:29.997726 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:29.997733 (+ 7us) server_negotiation.cc:407] Connection header received
0807 15:15:29.997774 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:29.997774 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:29.997794 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:29.997854 (+ 60us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 15:15:29.997950 (+ 96us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:29.998430 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:29.999205 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:29.999404 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:29.999456 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:29.999569 (+ 113us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 15:15:29.999673 (+ 104us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 15:15:29.999733 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0807 15:15:29.999746 (+ 13us) server_negotiation.cc:299] Negotiation successful
0807 15:15:29.999765 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260807 15:15:30.002115 24293 raft_consensus.cc:1261] T 46e5ff15c6854882b54fc91fd49b2f8c P 499d5bda369e428b9959486c40b077b3 [term 1 FOLLOWER]: Refusing update from remote peer 0c1c121d979444428857f8aa2612e077: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:15:30.002115 24728 raft_consensus.cc:1261] T 46e5ff15c6854882b54fc91fd49b2f8c P 39f290ad1d524320b20d18041dc24e7a [term 1 FOLLOWER]: Refusing update from remote peer 0c1c121d979444428857f8aa2612e077: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:15:30.002497 24845 consensus_queue.cc:1035] T 46e5ff15c6854882b54fc91fd49b2f8c P 0c1c121d979444428857f8aa2612e077 [LEADER]: Connected to new peer: Peer: permanent_uuid: "499d5bda369e428b9959486c40b077b3" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 44595 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:15:30.002662 24851 consensus_queue.cc:1035] T 46e5ff15c6854882b54fc91fd49b2f8c P 0c1c121d979444428857f8aa2612e077 [LEADER]: Connected to new peer: Peer: permanent_uuid: "39f290ad1d524320b20d18041dc24e7a" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 38471 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:15:30.004992 24859 mvcc.cc:205] Tried to move back new op lower bound from 7315930030084444160 to 7315930029975982080. Current Snapshot: MvccSnapshot[applied={T|T < 7315930030084444160}]
I20260807 15:15:30.005903 24863 mvcc.cc:205] Tried to move back new op lower bound from 7315930030084444160 to 7315930029975982080. Current Snapshot: MvccSnapshot[applied={T|T < 7315930030084444160 or (T in {7315930030084444160})}]
I20260807 15:15:30.006363 24860 mvcc.cc:205] Tried to move back new op lower bound from 7315930030084444160 to 7315930029975982080. Current Snapshot: MvccSnapshot[applied={T|T < 7315930030084444160 or (T in {7315930030084444160})}]
I20260807 15:15:30.007675 24071 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:48102:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 15:15:30.007781 24071 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:48102:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 15:15:30.008527 24071 catalog_manager.cc:5782] T 00000000000000000000000000000000 P caa589d19c1f43c7a2f1572b958619b7: Sending DeleteTablet for 3 replicas of tablet 46e5ff15c6854882b54fc91fd49b2f8c
I20260807 15:15:30.008883 24486 tablet_service.cc:1514] Processing DeleteTablet for tablet 46e5ff15c6854882b54fc91fd49b2f8c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:15:30 UTC) from {username='slave'} at 127.0.0.1:49172
I20260807 15:15:30.008865 24265 tablet_service.cc:1514] Processing DeleteTablet for tablet 46e5ff15c6854882b54fc91fd49b2f8c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:15:30 UTC) from {username='slave'} at 127.0.0.1:56506
I20260807 15:15:30.009136 24872 tablet_replica.cc:330] T 46e5ff15c6854882b54fc91fd49b2f8c P 0c1c121d979444428857f8aa2612e077: stopping tablet replica
I20260807 15:15:30.009119 24694 tablet_service.cc:1514] Processing DeleteTablet for tablet 46e5ff15c6854882b54fc91fd49b2f8c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:15:30 UTC) from {username='slave'} at 127.0.0.1:51118
I20260807 15:15:30.009239 24872 raft_consensus.cc:2229] T 46e5ff15c6854882b54fc91fd49b2f8c P 0c1c121d979444428857f8aa2612e077 [term 1 LEADER]: Raft consensus shutting down.
I20260807 15:15:30.009330 24872 raft_consensus.cc:2258] T 46e5ff15c6854882b54fc91fd49b2f8c P 0c1c121d979444428857f8aa2612e077 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:15:30.009390 30472 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7fZSIL/build/release/bin/kudu with pid 24166
I20260807 15:15:30.009409 24873 tablet_replica.cc:330] T 46e5ff15c6854882b54fc91fd49b2f8c P 39f290ad1d524320b20d18041dc24e7a: stopping tablet replica
I20260807 15:15:30.012121 24873 raft_consensus.cc:2229] T 46e5ff15c6854882b54fc91fd49b2f8c P 39f290ad1d524320b20d18041dc24e7a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 15:15:30.012256 24873 raft_consensus.cc:2258] T 46e5ff15c6854882b54fc91fd49b2f8c P 39f290ad1d524320b20d18041dc24e7a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:15:30.012264 24872 ts_tablet_manager.cc:1905] T 46e5ff15c6854882b54fc91fd49b2f8c P 0c1c121d979444428857f8aa2612e077: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:15:30.014025 24872 ts_tablet_manager.cc:1918] T 46e5ff15c6854882b54fc91fd49b2f8c P 0c1c121d979444428857f8aa2612e077: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 15:15:30.014096 24872 log.cc:1196] T 46e5ff15c6854882b54fc91fd49b2f8c P 0c1c121d979444428857f8aa2612e077: Deleting WAL directory at /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115480851507-30472-0/minicluster-data/ts-1/wal/wals/46e5ff15c6854882b54fc91fd49b2f8c
I20260807 15:15:30.014309 24872 ts_tablet_manager.cc:1939] T 46e5ff15c6854882b54fc91fd49b2f8c P 0c1c121d979444428857f8aa2612e077: Deleting consensus metadata
I20260807 15:15:30.015491 24050 catalog_manager.cc:4841] TS 0c1c121d979444428857f8aa2612e077 (127.29.194.2:38241): tablet 46e5ff15c6854882b54fc91fd49b2f8c (table test-table [id=c3908f06057b486c9b99336d00febf37]) successfully deleted
I20260807 15:15:30.018043 24873 ts_tablet_manager.cc:1905] T 46e5ff15c6854882b54fc91fd49b2f8c P 39f290ad1d524320b20d18041dc24e7a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:15:30.019553 24873 ts_tablet_manager.cc:1918] T 46e5ff15c6854882b54fc91fd49b2f8c P 39f290ad1d524320b20d18041dc24e7a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 15:15:30.019629 24873 log.cc:1196] T 46e5ff15c6854882b54fc91fd49b2f8c P 39f290ad1d524320b20d18041dc24e7a: Deleting WAL directory at /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115480851507-30472-0/minicluster-data/ts-2/wal/wals/46e5ff15c6854882b54fc91fd49b2f8c
I20260807 15:15:30.019848 24873 ts_tablet_manager.cc:1939] T 46e5ff15c6854882b54fc91fd49b2f8c P 39f290ad1d524320b20d18041dc24e7a: Deleting consensus metadata
I20260807 15:15:30.020336 24050 catalog_manager.cc:4841] TS 39f290ad1d524320b20d18041dc24e7a (127.29.194.3:38471): tablet 46e5ff15c6854882b54fc91fd49b2f8c (table test-table [id=c3908f06057b486c9b99336d00febf37]) successfully deleted
W20260807 15:15:30.024330 24051 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.29.194.1:44595 (error 108)
I20260807 15:15:30.024596 30472 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7fZSIL/build/release/bin/kudu with pid 24391
I20260807 15:15:30.027386 24830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:30.027217 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.1:44595 (local address 127.0.0.1:56518)
0807 15:15:30.027294 (+ 77us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:30.027346 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.194.1:44595: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260807 15:15:30.027521 24051 catalog_manager.cc:4584] TS 499d5bda369e428b9959486c40b077b3 (127.29.194.1:44595): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 46e5ff15c6854882b54fc91fd49b2f8c: Network error: Client connection negotiation failed: client connection to 127.29.194.1:44595: connect: Connection refused (error 111)
I20260807 15:15:30.031993 30472 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7fZSIL/build/release/bin/kudu with pid 24594
I20260807 15:15:30.043579 30472 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7fZSIL/build/release/bin/kudu with pid 24024
2026-08-07T15:15:30Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (820 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115480851507-30472-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 15:15:30 dist-test-slave-42z9 krb5kdc[24880](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 15:15:30 dist-test-slave-42z9 krb5kdc[24880](info): set up 2 sockets
Aug 07 15:15:30 dist-test-slave-42z9 krb5kdc[24880](info): commencing operation
krb5kdc: starting...
W20260807 15:15:31.197229 30472 mini_kdc.cc:121] Time spent starting KDC: real 1.127s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115480851507-30472-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115480851507-30472-0/krb5kdc/test-user.keytab.
Aug 07 15:15:31 dist-test-slave-42z9 krb5kdc[24880](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115731, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T15:15:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T15:15:31Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.194.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.194.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.194.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.62.keytab.
Entry for principal kudu/127.29.194.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.62.keytab.
I20260807 15:15:31.326419 30472 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7fZSIL/build/release/bin/kudu
/tmp/dist-test-task7fZSIL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115480851507-30472-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115480851507-30472-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115480851507-30472-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115480851507-30472-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.194.62:35603
--webserver_interface=127.29.194.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.62.keytab
--principal=kudu/127.29.194.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.194.20:33513
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.194.62:35603
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115480851507-30472-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115480851507-30472-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115480851507-30472-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:15:31.412755 24898 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:15:31.413009 24898 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:15:31.413035 24898 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:15:31.414359 24898 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:15:31.414412 24898 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:15:31.414426 24898 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:15:31.414439 24898 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:15:31.414450 24898 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:15:31.416035 24898 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.194.20:33513
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115480851507-30472-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115480851507-30472-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.194.62:35603
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.62.keytab
--principal=kudu/127.29.194.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.194.62:35603
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115480851507-30472-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.194.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24898
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115480851507-30472-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:15:31.416271 24898 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:15:31.416494 24898 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:15:31.421780 24906 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:15:31.421780 24907 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:15:31.421809 24909 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:15:31.422130 24898 server_base.cc:1034] running on GCE node
I20260807 15:15:31.422406 24898 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:15:31.422724 24898 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:15:31.423890 24898 hybrid_clock.cc:648] HybridClock initialized: now 1786115731423834 us; error 72 us; skew 500 ppm
Aug 07 15:15:31 dist-test-slave-42z9 krb5kdc[24880](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115731, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.194.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:15:31.431267 24898 init.cc:377] Logged in from keytab as kudu/127.29.194.62@KRBTEST.COM (short username kudu)
I20260807 15:15:31.431895 24898 webserver.cc:460] Webserver started at http://127.29.194.62:38471/ using document root <none> and password file <none>
I20260807 15:15:31.432122 24898 fs_manager.cc:359] Metadata directory not provided
I20260807 15:15:31.432173 24898 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:15:31.432312 24898 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:15:31.433425 24898 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115480851507-30472-0/minicluster-data/master-0/data/instance:
uuid: "b24a3d7d6c94417abcf6c7efd02fffe5"
format_stamp: "Formatted at 2026-08-07 15:15:31 on dist-test-slave-42z9"
server_key: "7e82b24e56fe7dc8c113249b67ca216d"
server_key_iv: "08bfb10f61fec4473fb82e926ae0c6ad"
server_key_version: "encryptionkey@0"
I20260807 15:15:31.433846 24898 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115480851507-30472-0/minicluster-data/master-0/wal/instance:
uuid: "b24a3d7d6c94417abcf6c7efd02fffe5"
format_stamp: "Formatted at 2026-08-07 15:15:31 on dist-test-slave-42z9"
server_key: "7e82b24e56fe7dc8c113249b67ca216d"
server_key_iv: "08bfb10f61fec4473fb82e926ae0c6ad"
server_key_version: "encryptionkey@0"
I20260807 15:15:31.435175 24898 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260807 15:15:31.435932 24916 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:15:31.436116 24898 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260807 15:15:31.436213 24898 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115480851507-30472-0/minicluster-data/master-0/data,/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115480851507-30472-0/minicluster-data/master-0/wal
uuid: "b24a3d7d6c94417abcf6c7efd02fffe5"
format_stamp: "Formatted at 2026-08-07 15:15:31 on dist-test-slave-42z9"
server_key: "7e82b24e56fe7dc8c113249b67ca216d"
server_key_iv: "08bfb10f61fec4473fb82e926ae0c6ad"
server_key_version: "encryptionkey@0"
I20260807 15:15:31.436295 24898 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115480851507-30472-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115480851507-30472-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115480851507-30472-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:15:31.455796 24898 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:15:31.458081 24898 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:15:31.458231 24898 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:15:31.462565 24898 rpc_server.cc:307] RPC server started. Bound to: 127.29.194.62:35603
I20260807 15:15:31.462692 24996 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.194.62:35603 every 8 connection(s)
I20260807 15:15:31.462965 24898 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115480851507-30472-0/minicluster-data/master-0/data/info.pb
I20260807 15:15:31.463951 25000 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:15:31.466475 25000 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b24a3d7d6c94417abcf6c7efd02fffe5: Bootstrap starting.
I20260807 15:15:31.467110 25000 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b24a3d7d6c94417abcf6c7efd02fffe5: Neither blocks nor log segments found. Creating new log.
I20260807 15:15:31.467401 25000 log.cc:824] T 00000000000000000000000000000000 P b24a3d7d6c94417abcf6c7efd02fffe5: Log is configured to *not* fsync() on all Append() calls
I20260807 15:15:31.468194 25000 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b24a3d7d6c94417abcf6c7efd02fffe5: No bootstrap required, opened a new log
I20260807 15:15:31.469933 25000 raft_consensus.cc:348] T 00000000000000000000000000000000 P b24a3d7d6c94417abcf6c7efd02fffe5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b24a3d7d6c94417abcf6c7efd02fffe5" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 35603 } }
I20260807 15:15:31.470081 25000 raft_consensus.cc:374] T 00000000000000000000000000000000 P b24a3d7d6c94417abcf6c7efd02fffe5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:15:31.470103 25000 raft_consensus.cc:729] T 00000000000000000000000000000000 P b24a3d7d6c94417abcf6c7efd02fffe5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b24a3d7d6c94417abcf6c7efd02fffe5, State: Initialized, Role: FOLLOWER
I20260807 15:15:31.470234 25000 consensus_queue.cc:260] T 00000000000000000000000000000000 P b24a3d7d6c94417abcf6c7efd02fffe5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b24a3d7d6c94417abcf6c7efd02fffe5" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 35603 } }
I20260807 15:15:31.470319 25000 raft_consensus.cc:388] T 00000000000000000000000000000000 P b24a3d7d6c94417abcf6c7efd02fffe5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:15:31.470347 25000 raft_consensus.cc:482] T 00000000000000000000000000000000 P b24a3d7d6c94417abcf6c7efd02fffe5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:15:31.470381 25000 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b24a3d7d6c94417abcf6c7efd02fffe5 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:15:31.470958 25000 raft_consensus.cc:504] T 00000000000000000000000000000000 P b24a3d7d6c94417abcf6c7efd02fffe5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b24a3d7d6c94417abcf6c7efd02fffe5" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 35603 } }
I20260807 15:15:31.471057 25000 leader_election.cc:302] T 00000000000000000000000000000000 P b24a3d7d6c94417abcf6c7efd02fffe5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b24a3d7d6c94417abcf6c7efd02fffe5; no voters:
I20260807 15:15:31.471231 25000 leader_election.cc:288] T 00000000000000000000000000000000 P b24a3d7d6c94417abcf6c7efd02fffe5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 15:15:31.471275 25003 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b24a3d7d6c94417abcf6c7efd02fffe5 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:15:31.471444 25003 raft_consensus.cc:686] T 00000000000000000000000000000000 P b24a3d7d6c94417abcf6c7efd02fffe5 [term 1 LEADER]: Becoming Leader. State: Replica: b24a3d7d6c94417abcf6c7efd02fffe5, State: Running, Role: LEADER
I20260807 15:15:31.471532 25000 sys_catalog.cc:564] T 00000000000000000000000000000000 P b24a3d7d6c94417abcf6c7efd02fffe5 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:15:31.471534 25003 consensus_queue.cc:237] T 00000000000000000000000000000000 P b24a3d7d6c94417abcf6c7efd02fffe5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b24a3d7d6c94417abcf6c7efd02fffe5" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 35603 } }
I20260807 15:15:31.471961 25006 sys_catalog.cc:455] T 00000000000000000000000000000000 P b24a3d7d6c94417abcf6c7efd02fffe5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b24a3d7d6c94417abcf6c7efd02fffe5. Latest consensus state: current_term: 1 leader_uuid: "b24a3d7d6c94417abcf6c7efd02fffe5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b24a3d7d6c94417abcf6c7efd02fffe5" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 35603 } } }
I20260807 15:15:31.472055 25006 sys_catalog.cc:458] T 00000000000000000000000000000000 P b24a3d7d6c94417abcf6c7efd02fffe5 [sys.catalog]: This master's current role is: LEADER
I20260807 15:15:31.472114 25005 sys_catalog.cc:455] T 00000000000000000000000000000000 P b24a3d7d6c94417abcf6c7efd02fffe5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b24a3d7d6c94417abcf6c7efd02fffe5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b24a3d7d6c94417abcf6c7efd02fffe5" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 35603 } } }
I20260807 15:15:31.472205 25005 sys_catalog.cc:458] T 00000000000000000000000000000000 P b24a3d7d6c94417abcf6c7efd02fffe5 [sys.catalog]: This master's current role is: LEADER
I20260807 15:15:31.472424 30472 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7fZSIL/build/release/bin/kudu as pid 24898
I20260807 15:15:31.472540 30472 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115480851507-30472-0/minicluster-data/master-0/wal/instance
I20260807 15:15:31.472780 30472 external_mini_cluster.cc:1428] Setting key 54a898647cd457e2eb390eb14de00b47
W20260807 15:15:31.473433 25024 catalog_manager.cc:1558] T 00000000000000000000000000000000 P b24a3d7d6c94417abcf6c7efd02fffe5: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 15:15:31.473532 25024 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 15:15:31.473590 25015 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:15:31.473769 25015 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:15:31.476296 25015 catalog_manager.cc:1347] Generated new cluster ID: f0c3d6b0d7cc4c588eaa16df81f06f90
I20260807 15:15:31.476358 25015 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 07 15:15:31 dist-test-slave-42z9 krb5kdc[24880](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115731, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.194.62@KRBTEST.COM
I20260807 15:15:31.481920 25015 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:15:31.482527 25015 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:15:31.484917 25029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:31.473270 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42936 (local address 127.29.194.62:35603)
0807 15:15:31.473585 (+ 315us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:31.473596 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:31.473629 (+ 33us) server_negotiation.cc:407] Connection header received
0807 15:15:31.473755 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:31.473763 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:31.473904 (+ 141us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:31.474106 (+ 202us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:15:31.474903 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:31.475873 (+ 970us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:31.476975 (+ 1102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:31.477654 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:31.477749 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:31.480651 (+ 2902us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:15:31.480655 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:15:31.480656 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:15:31.480669 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:15:31.483209 (+ 2540us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:15:31.483660 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:15:31.483664 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:15:31.483664 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:15:31.483790 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:15:31.483977 (+ 187us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:15:31.483978 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:15:31.483978 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:15:31.484278 (+ 300us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:15:31.484460 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0807 15:15:31.484531 (+ 71us) server_negotiation.cc:299] Negotiation successful
0807 15:15:31.484694 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":262,"thread_start_us":135,"threads_started":1}
I20260807 15:15:31.488677 25015 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b24a3d7d6c94417abcf6c7efd02fffe5: Generated new TSK 0
I20260807 15:15:31.488867 25015 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.29.194.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.194.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.194.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.1.keytab.
Entry for principal kudu/127.29.194.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.1.keytab.
I20260807 15:15:31.521405 30472 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7fZSIL/build/release/bin/kudu
/tmp/dist-test-task7fZSIL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115480851507-30472-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115480851507-30472-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115480851507-30472-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115480851507-30472-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.194.1:0
--local_ip_for_outbound_sockets=127.29.194.1
--webserver_interface=127.29.194.1
--webserver_port=0
--tserver_master_addrs=127.29.194.62:35603
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.1.keytab
--principal=kudu/127.29.194.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.194.20:33513
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115480851507-30472-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115480851507-30472-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115480851507-30472-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:15:31.609181 25034 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:15:31.609419 25034 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:15:31.609479 25034 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:15:31.610709 25034 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:15:31.610754 25034 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:15:31.610783 25034 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.194.1
I20260807 15:15:31.612741 25034 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.194.20:33513
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115480851507-30472-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115480851507-30472-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.1.keytab
--principal=kudu/127.29.194.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.194.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115480851507-30472-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.194.1
--webserver_port=0
--tserver_master_addrs=127.29.194.62:35603
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25034
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.194.1
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115480851507-30472-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:15:31.612990 25034 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:15:31.613209 25034 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:15:31.618444 25045 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:15:31.618464 25047 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:15:31.618613 25034 server_base.cc:1034] running on GCE node
W20260807 15:15:31.618469 25042 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:15:31.618916 25034 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:15:31.619177 25034 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:15:31.620350 25034 hybrid_clock.cc:648] HybridClock initialized: now 1786115731620331 us; error 36 us; skew 500 ppm
Aug 07 15:15:31 dist-test-slave-42z9 krb5kdc[24880](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115731, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.194.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:15:31.623013 25034 init.cc:377] Logged in from keytab as kudu/127.29.194.1@KRBTEST.COM (short username kudu)
I20260807 15:15:31.623821 25034 webserver.cc:460] Webserver started at http://127.29.194.1:35145/ using document root <none> and password file <none>
I20260807 15:15:31.624178 25034 fs_manager.cc:359] Metadata directory not provided
I20260807 15:15:31.624239 25034 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:15:31.624383 25034 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:15:31.625751 25034 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115480851507-30472-0/minicluster-data/ts-0/data/instance:
uuid: "a0c89a93e4ad4216887f7a3912b7c6ff"
format_stamp: "Formatted at 2026-08-07 15:15:31 on dist-test-slave-42z9"
server_key: "0a2764f8ac6cbd022496167b5d16ffb1"
server_key_iv: "c29ea248c6ba6521e4c1758875c5ca2a"
server_key_version: "encryptionkey@0"
I20260807 15:15:31.626121 25034 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115480851507-30472-0/minicluster-data/ts-0/wal/instance:
uuid: "a0c89a93e4ad4216887f7a3912b7c6ff"
format_stamp: "Formatted at 2026-08-07 15:15:31 on dist-test-slave-42z9"
server_key: "0a2764f8ac6cbd022496167b5d16ffb1"
server_key_iv: "c29ea248c6ba6521e4c1758875c5ca2a"
server_key_version: "encryptionkey@0"
I20260807 15:15:31.627928 25034 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260807 15:15:31.628981 25055 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:15:31.629196 25034 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:15:31.629309 25034 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115480851507-30472-0/minicluster-data/ts-0/data,/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115480851507-30472-0/minicluster-data/ts-0/wal
uuid: "a0c89a93e4ad4216887f7a3912b7c6ff"
format_stamp: "Formatted at 2026-08-07 15:15:31 on dist-test-slave-42z9"
server_key: "0a2764f8ac6cbd022496167b5d16ffb1"
server_key_iv: "c29ea248c6ba6521e4c1758875c5ca2a"
server_key_version: "encryptionkey@0"
I20260807 15:15:31.629393 25034 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115480851507-30472-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115480851507-30472-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115480851507-30472-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:15:31.637156 25034 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:15:31.639681 25034 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:15:31.639829 25034 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:15:31.640064 25034 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:15:31.640403 25034 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:15:31.640442 25034 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:15:31.640475 25034 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:15:31.640498 25034 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:15:31.646342 25034 rpc_server.cc:307] RPC server started. Bound to: 127.29.194.1:42503
I20260807 15:15:31.646389 25225 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.194.1:42503 every 8 connection(s)
I20260807 15:15:31.646729 25034 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115480851507-30472-0/minicluster-data/ts-0/data/info.pb
I20260807 15:15:31.647740 30472 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7fZSIL/build/release/bin/kudu as pid 25034
I20260807 15:15:31.647840 30472 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115480851507-30472-0/minicluster-data/ts-0/wal/instance
I20260807 15:15:31.648043 30472 external_mini_cluster.cc:1428] Setting key 200d4ed2864697280ebc3c51773cd59b
Aug 07 15:15:31 dist-test-slave-42z9 krb5kdc[24880](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115731, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.194.1@KRBTEST.COM for kudu/127.29.194.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.29.194.2@KRBTEST.COM; defaulting to no policy
I20260807 15:15:31.658259 25029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:31.647493 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.194.1:50935 (local address 127.29.194.62:35603)
0807 15:15:31.647575 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:31.647577 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:31.647777 (+ 200us) server_negotiation.cc:407] Connection header received
0807 15:15:31.648929 (+ 1152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:31.648930 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:31.648943 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:31.648998 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:15:31.650052 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:31.650765 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:31.652277 (+ 1512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:31.652493 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:31.652551 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:31.655436 (+ 2885us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:15:31.655437 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:15:31.655437 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:15:31.655444 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:15:31.656716 (+ 1272us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:15:31.657212 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:15:31.657213 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:15:31.657213 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:15:31.657270 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:15:31.657592 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:15:31.657592 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:15:31.657593 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:15:31.657704 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:15:31.657758 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0807 15:15:31.658110 (+ 352us) server_negotiation.cc:299] Negotiation successful
0807 15:15:31.658190 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260807 15:15:31.658514 25230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:31.647424 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:35603 (local address 127.29.194.1:50935)
0807 15:15:31.647645 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:31.647688 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0807 15:15:31.648782 (+ 1094us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:15:31.649077 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:15:31.649082 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:15:31.649712 (+ 630us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:15:31.649925 (+ 213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:31.649931 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:31.650903 (+ 972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:31.650904 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:31.652168 (+ 1264us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:31.652171 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:31.652671 (+ 500us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:31.652673 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:31.652972 (+ 299us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:31.653546 (+ 574us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:15:31.653557 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:15:31.655310 (+ 1753us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:15:31.656809 (+ 1499us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:15:31.656818 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:15:31.656821 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:15:31.657126 (+ 305us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:15:31.657333 (+ 207us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:15:31.657334 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:15:31.657334 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:15:31.657521 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:15:31.657772 (+ 251us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:15:31.657779 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:15:31.657964 (+ 185us) client_negotiation.cc:769] Sending connection context
0807 15:15:31.658078 (+ 114us) client_negotiation.cc:240] Negotiation successful
0807 15:15:31.658264 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":66,"threads_started":1}
I20260807 15:15:31.659185 25228 heartbeater.cc:347] Connected to a master server at 127.29.194.62:35603
I20260807 15:15:31.659296 25228 heartbeater.cc:464] Registering TS with master...
I20260807 15:15:31.659508 25228 heartbeater.cc:511] Master 127.29.194.62:35603 requested a full tablet report, sending...
I20260807 15:15:31.660169 24939 ts_manager.cc:194] Registered new tserver with Master: a0c89a93e4ad4216887f7a3912b7c6ff (127.29.194.1:42503)
I20260807 15:15:31.660907 24939 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.194.1@KRBTEST.COM'} at 127.29.194.1:50935
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.194.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.194.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.2.keytab.
Entry for principal kudu/127.29.194.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.2.keytab.
I20260807 15:15:31.676580 30472 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7fZSIL/build/release/bin/kudu
/tmp/dist-test-task7fZSIL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115480851507-30472-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115480851507-30472-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115480851507-30472-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115480851507-30472-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.194.2:0
--local_ip_for_outbound_sockets=127.29.194.2
--webserver_interface=127.29.194.2
--webserver_port=0
--tserver_master_addrs=127.29.194.62:35603
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.2.keytab
--principal=kudu/127.29.194.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.194.20:33513
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115480851507-30472-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115480851507-30472-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115480851507-30472-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:15:31.762096 25233 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:15:31.762315 25233 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:15:31.762339 25233 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:15:31.763455 25233 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:15:31.763492 25233 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:15:31.763525 25233 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.194.2
I20260807 15:15:31.764961 25233 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.194.20:33513
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115480851507-30472-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115480851507-30472-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.2.keytab
--principal=kudu/127.29.194.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.194.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115480851507-30472-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.194.2
--webserver_port=0
--tserver_master_addrs=127.29.194.62:35603
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25233
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.194.2
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115480851507-30472-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:15:31.765169 25233 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:15:31.765358 25233 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 15:15:31.770375 25233 server_base.cc:1034] running on GCE node
W20260807 15:15:31.770342 25240 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:15:31.770372 25244 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:15:31.770342 25242 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:15:31.770829 25233 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:15:31.771117 25233 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:15:31.772265 25233 hybrid_clock.cc:648] HybridClock initialized: now 1786115731772230 us; error 47 us; skew 500 ppm
Aug 07 15:15:31 dist-test-slave-42z9 krb5kdc[24880](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115731, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.194.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:15:31.774955 25233 init.cc:377] Logged in from keytab as kudu/127.29.194.2@KRBTEST.COM (short username kudu)
I20260807 15:15:31.775393 25233 webserver.cc:460] Webserver started at http://127.29.194.2:44069/ using document root <none> and password file <none>
I20260807 15:15:31.775597 25233 fs_manager.cc:359] Metadata directory not provided
I20260807 15:15:31.775637 25233 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:15:31.775730 25233 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:15:31.776705 25233 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115480851507-30472-0/minicluster-data/ts-1/data/instance:
uuid: "4239bb53709145e6bd46df919e25fa44"
format_stamp: "Formatted at 2026-08-07 15:15:31 on dist-test-slave-42z9"
server_key: "5108139859239dae86f4d528e663e9b0"
server_key_iv: "0d3b80f302497be4451cf430f12a4beb"
server_key_version: "encryptionkey@0"
I20260807 15:15:31.777004 25233 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115480851507-30472-0/minicluster-data/ts-1/wal/instance:
uuid: "4239bb53709145e6bd46df919e25fa44"
format_stamp: "Formatted at 2026-08-07 15:15:31 on dist-test-slave-42z9"
server_key: "5108139859239dae86f4d528e663e9b0"
server_key_iv: "0d3b80f302497be4451cf430f12a4beb"
server_key_version: "encryptionkey@0"
I20260807 15:15:31.778283 25233 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260807 15:15:31.778964 25252 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:15:31.779137 25233 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260807 15:15:31.779223 25233 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115480851507-30472-0/minicluster-data/ts-1/data,/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115480851507-30472-0/minicluster-data/ts-1/wal
uuid: "4239bb53709145e6bd46df919e25fa44"
format_stamp: "Formatted at 2026-08-07 15:15:31 on dist-test-slave-42z9"
server_key: "5108139859239dae86f4d528e663e9b0"
server_key_iv: "0d3b80f302497be4451cf430f12a4beb"
server_key_version: "encryptionkey@0"
I20260807 15:15:31.779284 25233 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115480851507-30472-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115480851507-30472-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115480851507-30472-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:15:31.794440 25233 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:15:31.796545 25233 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:15:31.796665 25233 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:15:31.796859 25233 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:15:31.797124 25233 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:15:31.797154 25233 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:15:31.797176 25233 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:15:31.797192 25233 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:15:31.802951 25233 rpc_server.cc:307] RPC server started. Bound to: 127.29.194.2:38635
I20260807 15:15:31.802995 25423 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.194.2:38635 every 8 connection(s)
I20260807 15:15:31.803306 25233 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115480851507-30472-0/minicluster-data/ts-1/data/info.pb
Aug 07 15:15:31 dist-test-slave-42z9 krb5kdc[24880](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115731, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.194.2@KRBTEST.COM for kudu/127.29.194.62@KRBTEST.COM
I20260807 15:15:31.813081 30472 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7fZSIL/build/release/bin/kudu as pid 25233
I20260807 15:15:31.813216 30472 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115480851507-30472-0/minicluster-data/ts-1/wal/instance
I20260807 15:15:31.813475 30472 external_mini_cluster.cc:1428] Setting key 7b2239b27309b784acdeff02cc49c39a
I20260807 15:15:31.813897 25029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:31.803997 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.194.2:60383 (local address 127.29.194.62:35603)
0807 15:15:31.804051 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:31.804052 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:31.804287 (+ 235us) server_negotiation.cc:407] Connection header received
0807 15:15:31.805168 (+ 881us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:31.805170 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:31.805181 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:31.805237 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:15:31.806296 (+ 1059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:31.806803 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:31.807893 (+ 1090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:31.808233 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:31.808320 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:31.810931 (+ 2611us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:15:31.810933 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:15:31.810933 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:15:31.810940 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:15:31.812053 (+ 1113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:15:31.812711 (+ 658us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:15:31.812712 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:15:31.812712 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:15:31.812777 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:15:31.813115 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:15:31.813116 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:15:31.813116 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:15:31.813284 (+ 168us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:15:31.813361 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0807 15:15:31.813714 (+ 353us) server_negotiation.cc:299] Negotiation successful
0807 15:15:31.813812 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260807 15:15:31.814134 25430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:31.803955 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:35603 (local address 127.29.194.2:60383)
0807 15:15:31.804114 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:31.804158 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0807 15:15:31.805069 (+ 911us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:15:31.805312 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:15:31.805315 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:15:31.805972 (+ 657us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:15:31.806175 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:31.806180 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:31.806912 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:31.806913 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:31.807739 (+ 826us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:31.807742 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:31.808411 (+ 669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:31.808412 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:31.808611 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:31.809316 (+ 705us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:15:31.809326 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:15:31.810807 (+ 1481us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:15:31.812145 (+ 1338us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:15:31.812153 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:15:31.812155 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:15:31.812578 (+ 423us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:15:31.812885 (+ 307us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:15:31.812887 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:15:31.812887 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:15:31.813023 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:15:31.813349 (+ 326us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:15:31.813355 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:15:31.813552 (+ 197us) client_negotiation.cc:769] Sending connection context
0807 15:15:31.813676 (+ 124us) client_negotiation.cc:240] Negotiation successful
0807 15:15:31.813878 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":81,"threads_started":1}
I20260807 15:15:31.814678 25425 heartbeater.cc:347] Connected to a master server at 127.29.194.62:35603
I20260807 15:15:31.814788 25425 heartbeater.cc:464] Registering TS with master...
I20260807 15:15:31.814991 25425 heartbeater.cc:511] Master 127.29.194.62:35603 requested a full tablet report, sending...
I20260807 15:15:31.815339 24939 ts_manager.cc:194] Registered new tserver with Master: 4239bb53709145e6bd46df919e25fa44 (127.29.194.2:38635)
I20260807 15:15:31.815755 24939 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.194.2@KRBTEST.COM'} at 127.29.194.2:60383
WARNING: no policy specified for kudu/127.29.194.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.194.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.194.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.3.keytab.
Entry for principal kudu/127.29.194.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.3.keytab.
I20260807 15:15:31.840547 30472 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7fZSIL/build/release/bin/kudu
/tmp/dist-test-task7fZSIL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115480851507-30472-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115480851507-30472-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115480851507-30472-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115480851507-30472-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.194.3:0
--local_ip_for_outbound_sockets=127.29.194.3
--webserver_interface=127.29.194.3
--webserver_port=0
--tserver_master_addrs=127.29.194.62:35603
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.3.keytab
--principal=kudu/127.29.194.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.194.20:33513
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115480851507-30472-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115480851507-30472-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115480851507-30472-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:15:31.924783 25438 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:15:31.925024 25438 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:15:31.925056 25438 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:15:31.926313 25438 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:15:31.926362 25438 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:15:31.926399 25438 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.194.3
I20260807 15:15:31.927950 25438 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.194.20:33513
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115480851507-30472-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115480851507-30472-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.3.keytab
--principal=kudu/127.29.194.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.194.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115480851507-30472-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.194.3
--webserver_port=0
--tserver_master_addrs=127.29.194.62:35603
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25438
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.194.3
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115480851507-30472-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:15:31.928236 25438 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:15:31.928478 25438 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:15:31.934160 25447 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:15:31.934237 25438 server_base.cc:1034] running on GCE node
W20260807 15:15:31.934160 25444 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:15:31.934163 25445 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:15:31.934581 25438 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:15:31.934863 25438 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:15:31.936012 25438 hybrid_clock.cc:648] HybridClock initialized: now 1786115731935995 us; error 32 us; skew 500 ppm
Aug 07 15:15:31 dist-test-slave-42z9 krb5kdc[24880](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115731, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.194.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:15:31.938544 25438 init.cc:377] Logged in from keytab as kudu/127.29.194.3@KRBTEST.COM (short username kudu)
I20260807 15:15:31.939034 25438 webserver.cc:460] Webserver started at http://127.29.194.3:46849/ using document root <none> and password file <none>
I20260807 15:15:31.939242 25438 fs_manager.cc:359] Metadata directory not provided
I20260807 15:15:31.939291 25438 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:15:31.939429 25438 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:15:31.940532 25438 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115480851507-30472-0/minicluster-data/ts-2/data/instance:
uuid: "421b5c46070146f8b61b9e8d20597011"
format_stamp: "Formatted at 2026-08-07 15:15:31 on dist-test-slave-42z9"
server_key: "a7f79abb98bd5f1d7f40ae1f73f50940"
server_key_iv: "ca12672188f18c38c20cadff08faefde"
server_key_version: "encryptionkey@0"
I20260807 15:15:31.940868 25438 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115480851507-30472-0/minicluster-data/ts-2/wal/instance:
uuid: "421b5c46070146f8b61b9e8d20597011"
format_stamp: "Formatted at 2026-08-07 15:15:31 on dist-test-slave-42z9"
server_key: "a7f79abb98bd5f1d7f40ae1f73f50940"
server_key_iv: "ca12672188f18c38c20cadff08faefde"
server_key_version: "encryptionkey@0"
I20260807 15:15:31.942262 25438 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260807 15:15:31.943236 25456 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:15:31.943503 25438 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:15:31.943622 25438 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115480851507-30472-0/minicluster-data/ts-2/data,/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115480851507-30472-0/minicluster-data/ts-2/wal
uuid: "421b5c46070146f8b61b9e8d20597011"
format_stamp: "Formatted at 2026-08-07 15:15:31 on dist-test-slave-42z9"
server_key: "a7f79abb98bd5f1d7f40ae1f73f50940"
server_key_iv: "ca12672188f18c38c20cadff08faefde"
server_key_version: "encryptionkey@0"
I20260807 15:15:31.943691 25438 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115480851507-30472-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115480851507-30472-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115480851507-30472-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:15:31.954067 25438 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:15:31.956035 25438 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:15:31.956176 25438 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:15:31.956638 25438 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:15:31.957161 25438 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:15:31.957208 25438 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:15:31.957240 25438 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:15:31.957263 25438 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:15:31.963217 25438 rpc_server.cc:307] RPC server started. Bound to: 127.29.194.3:36789
I20260807 15:15:31.963264 25623 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.194.3:36789 every 8 connection(s)
I20260807 15:15:31.963595 25438 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115480851507-30472-0/minicluster-data/ts-2/data/info.pb
I20260807 15:15:31.967111 30472 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7fZSIL/build/release/bin/kudu as pid 25438
I20260807 15:15:31.967222 30472 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115480851507-30472-0/minicluster-data/ts-2/wal/instance
I20260807 15:15:31.967401 30472 external_mini_cluster.cc:1428] Setting key 8dddb091b2977537556a843559df236a
Aug 07 15:15:31 dist-test-slave-42z9 krb5kdc[24880](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115731, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.194.3@KRBTEST.COM for kudu/127.29.194.62@KRBTEST.COM
I20260807 15:15:31.974579 25029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:31.964389 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.194.3:50269 (local address 127.29.194.62:35603)
0807 15:15:31.964495 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:31.964498 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:31.964773 (+ 275us) server_negotiation.cc:407] Connection header received
0807 15:15:31.965424 (+ 651us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:31.965426 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:31.965471 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:31.965560 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:15:31.966833 (+ 1273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:31.967546 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:31.968885 (+ 1339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:31.969145 (+ 260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:31.969235 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:31.971361 (+ 2126us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:15:31.971363 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:15:31.971363 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:15:31.971382 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:15:31.973155 (+ 1773us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:15:31.973654 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:15:31.973655 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:15:31.973655 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:15:31.973713 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:15:31.973913 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:15:31.973914 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:15:31.973914 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:15:31.974032 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:15:31.974080 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0807 15:15:31.974384 (+ 304us) server_negotiation.cc:299] Negotiation successful
0807 15:15:31.974491 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260807 15:15:31.974602 25627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:31.964285 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:35603 (local address 127.29.194.3:50269)
0807 15:15:31.964562 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:31.964640 (+ 78us) client_negotiation.cc:174] Beginning negotiation
0807 15:15:31.965301 (+ 661us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:15:31.965656 (+ 355us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:15:31.965660 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:15:31.966456 (+ 796us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:15:31.966692 (+ 236us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:31.966701 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:31.967649 (+ 948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:31.967651 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:31.968747 (+ 1096us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:31.968751 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:31.969239 (+ 488us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:31.969240 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:31.969344 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:31.969865 (+ 521us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:15:31.969873 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:15:31.971218 (+ 1345us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:15:31.973245 (+ 2027us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:15:31.973253 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:15:31.973255 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:15:31.973573 (+ 318us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:15:31.973763 (+ 190us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:15:31.973764 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:15:31.973764 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:15:31.973868 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:15:31.974071 (+ 203us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:15:31.974074 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:15:31.974197 (+ 123us) client_negotiation.cc:769] Sending connection context
0807 15:15:31.974278 (+ 81us) client_negotiation.cc:240] Negotiation successful
0807 15:15:31.974416 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":79,"threads_started":1}
I20260807 15:15:31.975170 25624 heartbeater.cc:347] Connected to a master server at 127.29.194.62:35603
I20260807 15:15:31.975287 25624 heartbeater.cc:464] Registering TS with master...
I20260807 15:15:31.975472 25624 heartbeater.cc:511] Master 127.29.194.62:35603 requested a full tablet report, sending...
I20260807 15:15:31.975850 24939 ts_manager.cc:194] Registered new tserver with Master: 421b5c46070146f8b61b9e8d20597011 (127.29.194.3:36789)
I20260807 15:15:31.976217 24939 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.194.3@KRBTEST.COM'} at 127.29.194.3:50269
I20260807 15:15:31.979121 30472 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 15:15:31.987017 25029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:31.980487 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42942 (local address 127.29.194.62:35603)
0807 15:15:31.980566 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:31.980568 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:31.980654 (+ 86us) server_negotiation.cc:407] Connection header received
0807 15:15:31.980720 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:31.980721 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:31.980731 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:31.980776 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:15:31.981334 (+ 558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:31.981912 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:31.982616 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:31.982817 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:31.982873 (+ 56us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:31.984174 (+ 1301us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:15:31.984175 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:15:31.984176 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:15:31.984183 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:15:31.985839 (+ 1656us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:15:31.986239 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:15:31.986240 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:15:31.986240 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:15:31.986293 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:15:31.986482 (+ 189us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:15:31.986483 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:15:31.986483 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:15:31.986620 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:15:31.986719 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0807 15:15:31.986852 (+ 133us) server_negotiation.cc:299] Negotiation successful
0807 15:15:31.986917 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260807 15:15:31.988628 24939 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42942:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 15:15:31.991261 24939 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 15:15:31.998507 25638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:31.994376 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38670 (local address 127.29.194.1:42503)
0807 15:15:31.994588 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:31.994593 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:31.994792 (+ 199us) server_negotiation.cc:407] Connection header received
0807 15:15:31.994904 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:31.994907 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:31.994921 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:31.994971 (+ 50us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:15:31.995183 (+ 212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:31.996347 (+ 1164us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:31.997540 (+ 1193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:31.998118 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:31.998185 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:31.998249 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0807 15:15:31.998383 (+ 134us) server_negotiation.cc:299] Negotiation successful
0807 15:15:31.998422 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":182,"thread_start_us":107,"threads_started":1}
I20260807 15:15:31.998502 25637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:31.994256 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.1:42503 (local address 127.0.0.1:38670)
0807 15:15:31.994737 (+ 481us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:31.994745 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0807 15:15:31.994816 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:15:31.995064 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:15:31.995064 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:15:31.995067 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:15:31.995106 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:31.995107 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:31.996465 (+ 1358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:31.996465 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:31.997394 (+ 929us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:31.997397 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:31.998197 (+ 800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:31.998198 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:31.998280 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:31.998294 (+ 14us) client_negotiation.cc:769] Sending connection context
0807 15:15:31.998359 (+ 65us) client_negotiation.cc:240] Negotiation successful
0807 15:15:31.998415 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":465,"mutex_wait_us":58,"thread_start_us":171,"threads_started":1}
I20260807 15:15:31.998679 25642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:31.994830 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52460 (local address 127.29.194.2:38635)
0807 15:15:31.995097 (+ 267us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:31.995101 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:31.995112 (+ 11us) server_negotiation.cc:407] Connection header received
0807 15:15:31.995153 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:31.995156 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:31.995170 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:31.995241 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:15:31.995379 (+ 138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:31.996068 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:31.997491 (+ 1423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:31.998167 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:31.998238 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:31.998293 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0807 15:15:31.998548 (+ 255us) server_negotiation.cc:299] Negotiation successful
0807 15:15:31.998597 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":233,"thread_start_us":86,"threads_started":1}
I20260807 15:15:31.998739 25636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:31.994256 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.3:36789 (local address 127.0.0.1:50016)
0807 15:15:31.994496 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:31.994540 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0807 15:15:31.994621 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:15:31.994932 (+ 311us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:15:31.994935 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:15:31.994938 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:15:31.995034 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:31.995036 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:31.996539 (+ 1503us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:31.996540 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:31.997681 (+ 1141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:31.997684 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:31.998545 (+ 861us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:31.998546 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:31.998601 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:31.998605 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 15:15:31.998643 (+ 38us) client_negotiation.cc:240] Negotiation successful
0807 15:15:31.998676 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":208,"mutex_wait_us":40,"thread_start_us":62,"threads_started":1}
I20260807 15:15:31.998888 25640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:31.994603 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.2:38635 (local address 127.0.0.1:52460)
0807 15:15:31.994828 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:31.994838 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 15:15:31.994894 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:15:31.995240 (+ 346us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:15:31.995241 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:15:31.995243 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:15:31.995301 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:31.995303 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:31.996161 (+ 858us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:31.996162 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:31.997383 (+ 1221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:31.997386 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:31.998331 (+ 945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:31.998332 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:31.998398 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:31.998403 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 15:15:31.998452 (+ 49us) client_negotiation.cc:240] Negotiation successful
0807 15:15:31.998499 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":202,"mutex_wait_us":1,"thread_start_us":69,"threads_started":1}
I20260807 15:15:31.999127 25639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:31.994549 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50016 (local address 127.29.194.3:36789)
0807 15:15:31.994740 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:31.994744 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:31.994755 (+ 11us) server_negotiation.cc:407] Connection header received
0807 15:15:31.994798 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:31.994800 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:31.994816 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:31.995111 (+ 295us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:15:31.995359 (+ 248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:31.996436 (+ 1077us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:31.997762 (+ 1326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:31.998458 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:31.998915 (+ 457us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:31.998982 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0807 15:15:31.999012 (+ 30us) server_negotiation.cc:299] Negotiation successful
0807 15:15:31.999054 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":150,"thread_start_us":110,"threads_started":1}
I20260807 15:15:31.999655 25322 tablet_service.cc:1467] Processing CreateTablet for tablet 30bde0f0e09748f2be0da4194bd48cee (DEFAULT_TABLE table=test-table [id=e50c3ecca24f4517858b375d83f16335]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:15:31.999655 25135 tablet_service.cc:1467] Processing CreateTablet for tablet 30bde0f0e09748f2be0da4194bd48cee (DEFAULT_TABLE table=test-table [id=e50c3ecca24f4517858b375d83f16335]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:15:31.999720 25530 tablet_service.cc:1467] Processing CreateTablet for tablet 30bde0f0e09748f2be0da4194bd48cee (DEFAULT_TABLE table=test-table [id=e50c3ecca24f4517858b375d83f16335]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:15:32.000046 25530 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 30bde0f0e09748f2be0da4194bd48cee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:15:32.000047 25135 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 30bde0f0e09748f2be0da4194bd48cee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:15:32.000047 25322 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 30bde0f0e09748f2be0da4194bd48cee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:15:32.003149 25644 tablet_bootstrap.cc:492] T 30bde0f0e09748f2be0da4194bd48cee P 421b5c46070146f8b61b9e8d20597011: Bootstrap starting.
I20260807 15:15:32.003855 25645 tablet_bootstrap.cc:492] T 30bde0f0e09748f2be0da4194bd48cee P 4239bb53709145e6bd46df919e25fa44: Bootstrap starting.
I20260807 15:15:32.004027 25644 tablet_bootstrap.cc:654] T 30bde0f0e09748f2be0da4194bd48cee P 421b5c46070146f8b61b9e8d20597011: Neither blocks nor log segments found. Creating new log.
I20260807 15:15:32.004146 25646 tablet_bootstrap.cc:492] T 30bde0f0e09748f2be0da4194bd48cee P a0c89a93e4ad4216887f7a3912b7c6ff: Bootstrap starting.
I20260807 15:15:32.004521 25645 tablet_bootstrap.cc:654] T 30bde0f0e09748f2be0da4194bd48cee P 4239bb53709145e6bd46df919e25fa44: Neither blocks nor log segments found. Creating new log.
I20260807 15:15:32.004740 25646 tablet_bootstrap.cc:654] T 30bde0f0e09748f2be0da4194bd48cee P a0c89a93e4ad4216887f7a3912b7c6ff: Neither blocks nor log segments found. Creating new log.
I20260807 15:15:32.004734 25644 log.cc:824] T 30bde0f0e09748f2be0da4194bd48cee P 421b5c46070146f8b61b9e8d20597011: Log is configured to *not* fsync() on all Append() calls
I20260807 15:15:32.005076 25645 log.cc:824] T 30bde0f0e09748f2be0da4194bd48cee P 4239bb53709145e6bd46df919e25fa44: Log is configured to *not* fsync() on all Append() calls
I20260807 15:15:32.005280 25646 log.cc:824] T 30bde0f0e09748f2be0da4194bd48cee P a0c89a93e4ad4216887f7a3912b7c6ff: Log is configured to *not* fsync() on all Append() calls
I20260807 15:15:32.005662 25644 tablet_bootstrap.cc:492] T 30bde0f0e09748f2be0da4194bd48cee P 421b5c46070146f8b61b9e8d20597011: No bootstrap required, opened a new log
I20260807 15:15:32.005723 25644 ts_tablet_manager.cc:1397] T 30bde0f0e09748f2be0da4194bd48cee P 421b5c46070146f8b61b9e8d20597011: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260807 15:15:32.005884 25645 tablet_bootstrap.cc:492] T 30bde0f0e09748f2be0da4194bd48cee P 4239bb53709145e6bd46df919e25fa44: No bootstrap required, opened a new log
I20260807 15:15:32.005955 25645 ts_tablet_manager.cc:1397] T 30bde0f0e09748f2be0da4194bd48cee P 4239bb53709145e6bd46df919e25fa44: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260807 15:15:32.006063 25646 tablet_bootstrap.cc:492] T 30bde0f0e09748f2be0da4194bd48cee P a0c89a93e4ad4216887f7a3912b7c6ff: No bootstrap required, opened a new log
I20260807 15:15:32.006136 25646 ts_tablet_manager.cc:1397] T 30bde0f0e09748f2be0da4194bd48cee P a0c89a93e4ad4216887f7a3912b7c6ff: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260807 15:15:32.007566 25645 raft_consensus.cc:348] T 30bde0f0e09748f2be0da4194bd48cee P 4239bb53709145e6bd46df919e25fa44 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "421b5c46070146f8b61b9e8d20597011" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 36789 } } peers { permanent_uuid: "a0c89a93e4ad4216887f7a3912b7c6ff" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 42503 } } peers { permanent_uuid: "4239bb53709145e6bd46df919e25fa44" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 38635 } }
I20260807 15:15:32.007715 25645 raft_consensus.cc:374] T 30bde0f0e09748f2be0da4194bd48cee P 4239bb53709145e6bd46df919e25fa44 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:15:32.007748 25645 raft_consensus.cc:729] T 30bde0f0e09748f2be0da4194bd48cee P 4239bb53709145e6bd46df919e25fa44 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4239bb53709145e6bd46df919e25fa44, State: Initialized, Role: FOLLOWER
I20260807 15:15:32.007668 25644 raft_consensus.cc:348] T 30bde0f0e09748f2be0da4194bd48cee P 421b5c46070146f8b61b9e8d20597011 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "421b5c46070146f8b61b9e8d20597011" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 36789 } } peers { permanent_uuid: "a0c89a93e4ad4216887f7a3912b7c6ff" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 42503 } } peers { permanent_uuid: "4239bb53709145e6bd46df919e25fa44" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 38635 } }
I20260807 15:15:32.007833 25644 raft_consensus.cc:374] T 30bde0f0e09748f2be0da4194bd48cee P 421b5c46070146f8b61b9e8d20597011 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:15:32.007874 25644 raft_consensus.cc:729] T 30bde0f0e09748f2be0da4194bd48cee P 421b5c46070146f8b61b9e8d20597011 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 421b5c46070146f8b61b9e8d20597011, State: Initialized, Role: FOLLOWER
I20260807 15:15:32.007882 25645 consensus_queue.cc:260] T 30bde0f0e09748f2be0da4194bd48cee P 4239bb53709145e6bd46df919e25fa44 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "421b5c46070146f8b61b9e8d20597011" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 36789 } } peers { permanent_uuid: "a0c89a93e4ad4216887f7a3912b7c6ff" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 42503 } } peers { permanent_uuid: "4239bb53709145e6bd46df919e25fa44" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 38635 } }
I20260807 15:15:32.007983 25644 consensus_queue.cc:260] T 30bde0f0e09748f2be0da4194bd48cee P 421b5c46070146f8b61b9e8d20597011 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "421b5c46070146f8b61b9e8d20597011" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 36789 } } peers { permanent_uuid: "a0c89a93e4ad4216887f7a3912b7c6ff" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 42503 } } peers { permanent_uuid: "4239bb53709145e6bd46df919e25fa44" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 38635 } }
I20260807 15:15:32.008165 25645 ts_tablet_manager.cc:1428] T 30bde0f0e09748f2be0da4194bd48cee P 4239bb53709145e6bd46df919e25fa44: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260807 15:15:32.008068 25646 raft_consensus.cc:348] T 30bde0f0e09748f2be0da4194bd48cee P a0c89a93e4ad4216887f7a3912b7c6ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "421b5c46070146f8b61b9e8d20597011" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 36789 } } peers { permanent_uuid: "a0c89a93e4ad4216887f7a3912b7c6ff" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 42503 } } peers { permanent_uuid: "4239bb53709145e6bd46df919e25fa44" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 38635 } }
I20260807 15:15:32.008219 25646 raft_consensus.cc:374] T 30bde0f0e09748f2be0da4194bd48cee P a0c89a93e4ad4216887f7a3912b7c6ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:15:32.008258 25644 ts_tablet_manager.cc:1428] T 30bde0f0e09748f2be0da4194bd48cee P 421b5c46070146f8b61b9e8d20597011: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260807 15:15:32.008270 25646 raft_consensus.cc:729] T 30bde0f0e09748f2be0da4194bd48cee P a0c89a93e4ad4216887f7a3912b7c6ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0c89a93e4ad4216887f7a3912b7c6ff, State: Initialized, Role: FOLLOWER
I20260807 15:15:32.008358 25624 heartbeater.cc:503] Master 127.29.194.62:35603 was elected leader, sending a full tablet report...
I20260807 15:15:32.008370 25646 consensus_queue.cc:260] T 30bde0f0e09748f2be0da4194bd48cee P a0c89a93e4ad4216887f7a3912b7c6ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "421b5c46070146f8b61b9e8d20597011" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 36789 } } peers { permanent_uuid: "a0c89a93e4ad4216887f7a3912b7c6ff" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 42503 } } peers { permanent_uuid: "4239bb53709145e6bd46df919e25fa44" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 38635 } }
I20260807 15:15:32.008639 25646 ts_tablet_manager.cc:1428] T 30bde0f0e09748f2be0da4194bd48cee P a0c89a93e4ad4216887f7a3912b7c6ff: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260807 15:15:32.008651 25228 heartbeater.cc:503] Master 127.29.194.62:35603 was elected leader, sending a full tablet report...
I20260807 15:15:32.008764 25425 heartbeater.cc:503] Master 127.29.194.62:35603 was elected leader, sending a full tablet report...
W20260807 15:15:32.053886 25428 tablet.cc:2369] T 30bde0f0e09748f2be0da4194bd48cee P 4239bb53709145e6bd46df919e25fa44: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 15:15:32.147507 25229 tablet.cc:2369] T 30bde0f0e09748f2be0da4194bd48cee P a0c89a93e4ad4216887f7a3912b7c6ff: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 15:15:32.166599 25654 raft_consensus.cc:482] T 30bde0f0e09748f2be0da4194bd48cee P a0c89a93e4ad4216887f7a3912b7c6ff [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 15:15:32.166762 25654 raft_consensus.cc:504] T 30bde0f0e09748f2be0da4194bd48cee P a0c89a93e4ad4216887f7a3912b7c6ff [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "421b5c46070146f8b61b9e8d20597011" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 36789 } } peers { permanent_uuid: "a0c89a93e4ad4216887f7a3912b7c6ff" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 42503 } } peers { permanent_uuid: "4239bb53709145e6bd46df919e25fa44" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 38635 } }
I20260807 15:15:32.167128 25654 leader_election.cc:288] T 30bde0f0e09748f2be0da4194bd48cee P a0c89a93e4ad4216887f7a3912b7c6ff [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 421b5c46070146f8b61b9e8d20597011 (127.29.194.3:36789), 4239bb53709145e6bd46df919e25fa44 (127.29.194.2:38635)
I20260807 15:15:32.170105 25639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:32.167506 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.194.1:45017 (local address 127.29.194.3:36789)
0807 15:15:32.167554 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:32.167556 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:32.167568 (+ 12us) server_negotiation.cc:407] Connection header received
0807 15:15:32.167653 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:32.167654 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:32.167664 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:32.167708 (+ 44us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:15:32.167863 (+ 155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:32.168323 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:32.169347 (+ 1024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:32.169777 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:32.169843 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:32.169852 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0807 15:15:32.169996 (+ 144us) server_negotiation.cc:299] Negotiation successful
0807 15:15:32.170033 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260807 15:15:32.170109 25642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:32.167396 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.194.1:37057 (local address 127.29.194.2:38635)
0807 15:15:32.167444 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:32.167446 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:32.167528 (+ 82us) server_negotiation.cc:407] Connection header received
0807 15:15:32.167608 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:32.167609 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:32.167618 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:32.167667 (+ 49us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:15:32.167813 (+ 146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:32.168315 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:32.169322 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:32.169786 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:32.169859 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:32.169867 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0807 15:15:32.170006 (+ 139us) server_negotiation.cc:299] Negotiation successful
0807 15:15:32.170042 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260807 15:15:32.170126 25657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:32.167336 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.3:36789 (local address 127.29.194.1:45017)
0807 15:15:32.167505 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:32.167515 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 15:15:32.167586 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:15:32.167721 (+ 135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:15:32.167722 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:15:32.167724 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:15:32.167800 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:32.167802 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:32.168401 (+ 599us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:32.168401 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:32.169254 (+ 853us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:32.169256 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:32.169854 (+ 598us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:32.169855 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:32.169934 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:32.169938 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 15:15:32.169984 (+ 46us) client_negotiation.cc:240] Negotiation successful
0807 15:15:32.170027 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154,"mutex_wait_us":58,"thread_start_us":64,"threads_started":1}
I20260807 15:15:32.170138 25655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:32.167326 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.2:38635 (local address 127.29.194.1:37057)
0807 15:15:32.167452 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:32.167463 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 15:15:32.167535 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:15:32.167675 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:15:32.167676 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:15:32.167678 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:15:32.167756 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:32.167758 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:32.168395 (+ 637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:32.168397 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:32.169233 (+ 836us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:32.169236 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:32.169862 (+ 626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:32.169863 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:32.169946 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:32.169949 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 15:15:32.169995 (+ 46us) client_negotiation.cc:240] Negotiation successful
0807 15:15:32.170039 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107,"mutex_wait_us":1,"thread_start_us":79,"threads_started":1}
I20260807 15:15:32.170735 25359 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "30bde0f0e09748f2be0da4194bd48cee" candidate_uuid: "a0c89a93e4ad4216887f7a3912b7c6ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4239bb53709145e6bd46df919e25fa44" is_pre_election: true
I20260807 15:15:32.170740 25557 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "30bde0f0e09748f2be0da4194bd48cee" candidate_uuid: "a0c89a93e4ad4216887f7a3912b7c6ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "421b5c46070146f8b61b9e8d20597011" is_pre_election: true
I20260807 15:15:32.170886 25557 raft_consensus.cc:2454] T 30bde0f0e09748f2be0da4194bd48cee P 421b5c46070146f8b61b9e8d20597011 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0c89a93e4ad4216887f7a3912b7c6ff in term 0.
I20260807 15:15:32.170904 25359 raft_consensus.cc:2454] T 30bde0f0e09748f2be0da4194bd48cee P 4239bb53709145e6bd46df919e25fa44 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0c89a93e4ad4216887f7a3912b7c6ff in term 0.
I20260807 15:15:32.171140 25059 leader_election.cc:302] T 30bde0f0e09748f2be0da4194bd48cee P a0c89a93e4ad4216887f7a3912b7c6ff [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 421b5c46070146f8b61b9e8d20597011, a0c89a93e4ad4216887f7a3912b7c6ff; no voters:
I20260807 15:15:32.171286 25654 raft_consensus.cc:2781] T 30bde0f0e09748f2be0da4194bd48cee P a0c89a93e4ad4216887f7a3912b7c6ff [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 15:15:32.171352 25654 raft_consensus.cc:482] T 30bde0f0e09748f2be0da4194bd48cee P a0c89a93e4ad4216887f7a3912b7c6ff [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 15:15:32.171368 25654 raft_consensus.cc:3037] T 30bde0f0e09748f2be0da4194bd48cee P a0c89a93e4ad4216887f7a3912b7c6ff [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:15:32.172175 25654 raft_consensus.cc:504] T 30bde0f0e09748f2be0da4194bd48cee P a0c89a93e4ad4216887f7a3912b7c6ff [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "421b5c46070146f8b61b9e8d20597011" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 36789 } } peers { permanent_uuid: "a0c89a93e4ad4216887f7a3912b7c6ff" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 42503 } } peers { permanent_uuid: "4239bb53709145e6bd46df919e25fa44" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 38635 } }
I20260807 15:15:32.172305 25654 leader_election.cc:288] T 30bde0f0e09748f2be0da4194bd48cee P a0c89a93e4ad4216887f7a3912b7c6ff [CANDIDATE]: Term 1 election: Requested vote from peers 421b5c46070146f8b61b9e8d20597011 (127.29.194.3:36789), 4239bb53709145e6bd46df919e25fa44 (127.29.194.2:38635)
I20260807 15:15:32.172458 25557 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "30bde0f0e09748f2be0da4194bd48cee" candidate_uuid: "a0c89a93e4ad4216887f7a3912b7c6ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "421b5c46070146f8b61b9e8d20597011"
I20260807 15:15:32.172549 25557 raft_consensus.cc:3037] T 30bde0f0e09748f2be0da4194bd48cee P 421b5c46070146f8b61b9e8d20597011 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:15:32.172572 25359 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "30bde0f0e09748f2be0da4194bd48cee" candidate_uuid: "a0c89a93e4ad4216887f7a3912b7c6ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4239bb53709145e6bd46df919e25fa44"
I20260807 15:15:32.172642 25359 raft_consensus.cc:3037] T 30bde0f0e09748f2be0da4194bd48cee P 4239bb53709145e6bd46df919e25fa44 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:15:32.173317 25557 raft_consensus.cc:2454] T 30bde0f0e09748f2be0da4194bd48cee P 421b5c46070146f8b61b9e8d20597011 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0c89a93e4ad4216887f7a3912b7c6ff in term 1.
I20260807 15:15:32.173406 25359 raft_consensus.cc:2454] T 30bde0f0e09748f2be0da4194bd48cee P 4239bb53709145e6bd46df919e25fa44 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0c89a93e4ad4216887f7a3912b7c6ff in term 1.
I20260807 15:15:32.173498 25059 leader_election.cc:302] T 30bde0f0e09748f2be0da4194bd48cee P a0c89a93e4ad4216887f7a3912b7c6ff [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 421b5c46070146f8b61b9e8d20597011, a0c89a93e4ad4216887f7a3912b7c6ff; no voters:
I20260807 15:15:32.173594 25654 raft_consensus.cc:2781] T 30bde0f0e09748f2be0da4194bd48cee P a0c89a93e4ad4216887f7a3912b7c6ff [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:15:32.173763 25654 raft_consensus.cc:686] T 30bde0f0e09748f2be0da4194bd48cee P a0c89a93e4ad4216887f7a3912b7c6ff [term 1 LEADER]: Becoming Leader. State: Replica: a0c89a93e4ad4216887f7a3912b7c6ff, State: Running, Role: LEADER
I20260807 15:15:32.173857 25654 consensus_queue.cc:237] T 30bde0f0e09748f2be0da4194bd48cee P a0c89a93e4ad4216887f7a3912b7c6ff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "421b5c46070146f8b61b9e8d20597011" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 36789 } } peers { permanent_uuid: "a0c89a93e4ad4216887f7a3912b7c6ff" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 42503 } } peers { permanent_uuid: "4239bb53709145e6bd46df919e25fa44" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 38635 } }
I20260807 15:15:32.174924 24938 catalog_manager.cc:5495] T 30bde0f0e09748f2be0da4194bd48cee P a0c89a93e4ad4216887f7a3912b7c6ff reported cstate change: term changed from 0 to 1, leader changed from <none> to a0c89a93e4ad4216887f7a3912b7c6ff (127.29.194.1). New cstate: current_term: 1 leader_uuid: "a0c89a93e4ad4216887f7a3912b7c6ff" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "421b5c46070146f8b61b9e8d20597011" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 36789 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a0c89a93e4ad4216887f7a3912b7c6ff" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 42503 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4239bb53709145e6bd46df919e25fa44" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 38635 } health_report { overall_health: UNKNOWN } } }
I20260807 15:15:32.186059 25638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:32.183186 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38684 (local address 127.29.194.1:42503)
0807 15:15:32.183249 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:32.183250 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:32.183259 (+ 9us) server_negotiation.cc:407] Connection header received
0807 15:15:32.183296 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:32.183297 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:32.183320 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:32.183403 (+ 83us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 15:15:32.183529 (+ 126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:32.183993 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:32.184836 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:32.185038 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:32.185099 (+ 61us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:32.185604 (+ 505us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 15:15:32.185717 (+ 113us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 15:15:32.185820 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0807 15:15:32.185957 (+ 137us) server_negotiation.cc:299] Negotiation successful
0807 15:15:32.185971 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260807 15:15:32.188017 25557 raft_consensus.cc:1261] T 30bde0f0e09748f2be0da4194bd48cee P 421b5c46070146f8b61b9e8d20597011 [term 1 FOLLOWER]: Refusing update from remote peer a0c89a93e4ad4216887f7a3912b7c6ff: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:15:32.188025 25359 raft_consensus.cc:1261] T 30bde0f0e09748f2be0da4194bd48cee P 4239bb53709145e6bd46df919e25fa44 [term 1 FOLLOWER]: Refusing update from remote peer a0c89a93e4ad4216887f7a3912b7c6ff: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:15:32.188352 25654 consensus_queue.cc:1035] T 30bde0f0e09748f2be0da4194bd48cee P a0c89a93e4ad4216887f7a3912b7c6ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "4239bb53709145e6bd46df919e25fa44" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 38635 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:15:32.188500 25658 consensus_queue.cc:1035] T 30bde0f0e09748f2be0da4194bd48cee P a0c89a93e4ad4216887f7a3912b7c6ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "421b5c46070146f8b61b9e8d20597011" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 36789 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:15:32.191097 25663 mvcc.cc:205] Tried to move back new op lower bound from 7315930039039426560 to 7315930038984757248. Current Snapshot: MvccSnapshot[applied={T|T < 7315930039039426560}]
I20260807 15:15:32.192135 25664 mvcc.cc:205] Tried to move back new op lower bound from 7315930039039426560 to 7315930038984757248. Current Snapshot: MvccSnapshot[applied={T|T < 7315930039039426560}]
I20260807 15:15:32.193233 24939 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42942:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 15:15:32.193320 24939 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42942:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 15:15:32.194211 24939 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b24a3d7d6c94417abcf6c7efd02fffe5: Sending DeleteTablet for 3 replicas of tablet 30bde0f0e09748f2be0da4194bd48cee
I20260807 15:15:32.194547 25530 tablet_service.cc:1514] Processing DeleteTablet for tablet 30bde0f0e09748f2be0da4194bd48cee with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:15:32 UTC) from {username='kudu', principal='kudu/127.29.194.62@KRBTEST.COM'} at 127.0.0.1:50016
I20260807 15:15:32.194626 25322 tablet_service.cc:1514] Processing DeleteTablet for tablet 30bde0f0e09748f2be0da4194bd48cee with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:15:32 UTC) from {username='kudu', principal='kudu/127.29.194.62@KRBTEST.COM'} at 127.0.0.1:52460
I20260807 15:15:32.194854 25674 tablet_replica.cc:330] T 30bde0f0e09748f2be0da4194bd48cee P 4239bb53709145e6bd46df919e25fa44: stopping tablet replica
I20260807 15:15:32.194828 25135 tablet_service.cc:1514] Processing DeleteTablet for tablet 30bde0f0e09748f2be0da4194bd48cee with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:15:32 UTC) from {username='kudu', principal='kudu/127.29.194.62@KRBTEST.COM'} at 127.0.0.1:38670
I20260807 15:15:32.194955 25674 raft_consensus.cc:2229] T 30bde0f0e09748f2be0da4194bd48cee P 4239bb53709145e6bd46df919e25fa44 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 15:15:32.195065 25674 raft_consensus.cc:2258] T 30bde0f0e09748f2be0da4194bd48cee P 4239bb53709145e6bd46df919e25fa44 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:15:32.195080 25677 tablet_replica.cc:330] T 30bde0f0e09748f2be0da4194bd48cee P a0c89a93e4ad4216887f7a3912b7c6ff: stopping tablet replica
I20260807 15:15:32.195196 25677 raft_consensus.cc:2229] T 30bde0f0e09748f2be0da4194bd48cee P a0c89a93e4ad4216887f7a3912b7c6ff [term 1 LEADER]: Raft consensus shutting down.
I20260807 15:15:32.195288 30472 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7fZSIL/build/release/bin/kudu with pid 25034
I20260807 15:15:32.195398 25677 raft_consensus.cc:2258] T 30bde0f0e09748f2be0da4194bd48cee P a0c89a93e4ad4216887f7a3912b7c6ff [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:15:32.195565 25674 ts_tablet_manager.cc:1905] T 30bde0f0e09748f2be0da4194bd48cee P 4239bb53709145e6bd46df919e25fa44: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:15:32.198246 25674 ts_tablet_manager.cc:1918] T 30bde0f0e09748f2be0da4194bd48cee P 4239bb53709145e6bd46df919e25fa44: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 15:15:32.198395 25674 log.cc:1196] T 30bde0f0e09748f2be0da4194bd48cee P 4239bb53709145e6bd46df919e25fa44: Deleting WAL directory at /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115480851507-30472-0/minicluster-data/ts-1/wal/wals/30bde0f0e09748f2be0da4194bd48cee
I20260807 15:15:32.198783 25674 ts_tablet_manager.cc:1939] T 30bde0f0e09748f2be0da4194bd48cee P 4239bb53709145e6bd46df919e25fa44: Deleting consensus metadata
I20260807 15:15:32.199297 24923 catalog_manager.cc:4841] TS 4239bb53709145e6bd46df919e25fa44 (127.29.194.2:38635): tablet 30bde0f0e09748f2be0da4194bd48cee (table test-table [id=e50c3ecca24f4517858b375d83f16335]) successfully deleted
I20260807 15:15:32.200753 25673 tablet_replica.cc:330] T 30bde0f0e09748f2be0da4194bd48cee P 421b5c46070146f8b61b9e8d20597011: stopping tablet replica
I20260807 15:15:32.200901 25673 raft_consensus.cc:2229] T 30bde0f0e09748f2be0da4194bd48cee P 421b5c46070146f8b61b9e8d20597011 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 15:15:32.201020 25673 raft_consensus.cc:2258] T 30bde0f0e09748f2be0da4194bd48cee P 421b5c46070146f8b61b9e8d20597011 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:15:32.201416 25673 ts_tablet_manager.cc:1905] T 30bde0f0e09748f2be0da4194bd48cee P 421b5c46070146f8b61b9e8d20597011: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:15:32.202806 25673 ts_tablet_manager.cc:1918] T 30bde0f0e09748f2be0da4194bd48cee P 421b5c46070146f8b61b9e8d20597011: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 15:15:32.202877 25673 log.cc:1196] T 30bde0f0e09748f2be0da4194bd48cee P 421b5c46070146f8b61b9e8d20597011: Deleting WAL directory at /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115480851507-30472-0/minicluster-data/ts-2/wal/wals/30bde0f0e09748f2be0da4194bd48cee
I20260807 15:15:32.203137 25673 ts_tablet_manager.cc:1939] T 30bde0f0e09748f2be0da4194bd48cee P 421b5c46070146f8b61b9e8d20597011: Deleting consensus metadata
I20260807 15:15:32.203580 24919 catalog_manager.cc:4841] TS 421b5c46070146f8b61b9e8d20597011 (127.29.194.3:36789): tablet 30bde0f0e09748f2be0da4194bd48cee (table test-table [id=e50c3ecca24f4517858b375d83f16335]) successfully deleted
W20260807 15:15:32.205281 24923 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.29.194.1:42503 (error 108)
I20260807 15:15:32.205771 30472 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7fZSIL/build/release/bin/kudu with pid 25233
I20260807 15:15:32.205909 25640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:32.205658 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.1:42503 (local address 127.0.0.1:38694)
0807 15:15:32.205771 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:32.205849 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.194.1:42503: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
W20260807 15:15:32.212913 24923 catalog_manager.cc:4584] TS a0c89a93e4ad4216887f7a3912b7c6ff (127.29.194.1:42503): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 30bde0f0e09748f2be0da4194bd48cee: Network error: Client connection negotiation failed: client connection to 127.29.194.1:42503: connect: Connection refused (error 111)
I20260807 15:15:32.227232 30472 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7fZSIL/build/release/bin/kudu with pid 25438
I20260807 15:15:32.240811 30472 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7fZSIL/build/release/bin/kudu with pid 24898
2026-08-07T15:15:32Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (2202 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-07T15:15:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T15:15:32Z Disabled control of system clock
I20260807 15:15:32.281100 30472 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7fZSIL/build/release/bin/kudu
/tmp/dist-test-task7fZSIL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115480851507-30472-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115480851507-30472-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115480851507-30472-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115480851507-30472-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.194.62:46753
--webserver_interface=127.29.194.62
--webserver_port=0
--builtin_ntp_servers=127.29.194.20:37653
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.194.62:46753
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260807 15:15:32.366474 25683 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:15:32.366714 25683 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:15:32.366735 25683 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:15:32.367947 25683 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:15:32.368000 25683 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:15:32.368021 25683 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:15:32.368031 25683 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:15:32.368043 25683 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:15:32.369539 25683 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.194.20:37653
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115480851507-30472-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115480851507-30472-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.194.62:46753
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.194.62:46753
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115480851507-30472-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.194.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25683
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115480851507-30472-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:15:32.369805 25683 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:15:32.370110 25683 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:15:32.375420 25691 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:15:32.375553 25692 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:15:32.375586 25683 server_base.cc:1034] running on GCE node
W20260807 15:15:32.375440 25694 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:15:32.375864 25683 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:15:32.376129 25683 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:15:32.377270 25683 hybrid_clock.cc:648] HybridClock initialized: now 1786115732377247 us; error 51 us; skew 500 ppm
I20260807 15:15:32.378638 25683 webserver.cc:460] Webserver started at http://127.29.194.62:46311/ using document root <none> and password file <none>
I20260807 15:15:32.378891 25683 fs_manager.cc:359] Metadata directory not provided
I20260807 15:15:32.378944 25683 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:15:32.379083 25683 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:15:32.380172 25683 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115480851507-30472-0/minicluster-data/master-0/data/instance:
uuid: "8d35796ccac14de3976172530653f7b9"
format_stamp: "Formatted at 2026-08-07 15:15:32 on dist-test-slave-42z9"
server_key: "ceaef601886e24e03f928ea701b186e4"
server_key_iv: "6f067c893df5cbf876872a8a530bf395"
server_key_version: "encryptionkey@0"
I20260807 15:15:32.380522 25683 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115480851507-30472-0/minicluster-data/master-0/wal/instance:
uuid: "8d35796ccac14de3976172530653f7b9"
format_stamp: "Formatted at 2026-08-07 15:15:32 on dist-test-slave-42z9"
server_key: "ceaef601886e24e03f928ea701b186e4"
server_key_iv: "6f067c893df5cbf876872a8a530bf395"
server_key_version: "encryptionkey@0"
I20260807 15:15:32.382474 25683 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.002s
I20260807 15:15:32.383323 25707 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:15:32.383920 25683 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:15:32.384027 25683 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115480851507-30472-0/minicluster-data/master-0/data,/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115480851507-30472-0/minicluster-data/master-0/wal
uuid: "8d35796ccac14de3976172530653f7b9"
format_stamp: "Formatted at 2026-08-07 15:15:32 on dist-test-slave-42z9"
server_key: "ceaef601886e24e03f928ea701b186e4"
server_key_iv: "6f067c893df5cbf876872a8a530bf395"
server_key_version: "encryptionkey@0"
I20260807 15:15:32.384099 25683 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115480851507-30472-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115480851507-30472-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115480851507-30472-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:15:32.399614 25683 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:15:32.399953 25683 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:15:32.400085 25683 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:15:32.404580 25683 rpc_server.cc:307] RPC server started. Bound to: 127.29.194.62:46753
I20260807 15:15:32.404626 25783 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.194.62:46753 every 8 connection(s)
I20260807 15:15:32.404987 25683 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115480851507-30472-0/minicluster-data/master-0/data/info.pb
I20260807 15:15:32.406090 25785 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:15:32.408217 30472 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7fZSIL/build/release/bin/kudu as pid 25683
I20260807 15:15:32.408332 30472 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115480851507-30472-0/minicluster-data/master-0/wal/instance
I20260807 15:15:32.408551 30472 external_mini_cluster.cc:1428] Setting key e484dc2ba2440eca15b8a48d2b9bacce
I20260807 15:15:32.408702 25785 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8d35796ccac14de3976172530653f7b9: Bootstrap starting.
I20260807 15:15:32.409677 25785 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8d35796ccac14de3976172530653f7b9: Neither blocks nor log segments found. Creating new log.
I20260807 15:15:32.410179 25785 log.cc:824] T 00000000000000000000000000000000 P 8d35796ccac14de3976172530653f7b9: Log is configured to *not* fsync() on all Append() calls
I20260807 15:15:32.411067 25785 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8d35796ccac14de3976172530653f7b9: No bootstrap required, opened a new log
I20260807 15:15:32.413264 25785 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8d35796ccac14de3976172530653f7b9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8d35796ccac14de3976172530653f7b9" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 46753 } }
I20260807 15:15:32.413483 25785 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8d35796ccac14de3976172530653f7b9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:15:32.413535 25785 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8d35796ccac14de3976172530653f7b9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8d35796ccac14de3976172530653f7b9, State: Initialized, Role: FOLLOWER
I20260807 15:15:32.413661 25785 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8d35796ccac14de3976172530653f7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8d35796ccac14de3976172530653f7b9" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 46753 } }
I20260807 15:15:32.413740 25785 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8d35796ccac14de3976172530653f7b9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:15:32.413779 25785 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8d35796ccac14de3976172530653f7b9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:15:32.413825 25785 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8d35796ccac14de3976172530653f7b9 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:15:32.414422 25785 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8d35796ccac14de3976172530653f7b9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8d35796ccac14de3976172530653f7b9" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 46753 } }
I20260807 15:15:32.414549 25785 leader_election.cc:302] T 00000000000000000000000000000000 P 8d35796ccac14de3976172530653f7b9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8d35796ccac14de3976172530653f7b9; no voters:
I20260807 15:15:32.414608 25792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:32.409118 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48886 (local address 127.29.194.62:46753)
0807 15:15:32.409327 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:32.409334 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:32.409359 (+ 25us) server_negotiation.cc:407] Connection header received
0807 15:15:32.409860 (+ 501us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:32.409867 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:32.409930 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:32.410051 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:15:32.410539 (+ 488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:32.411553 (+ 1014us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:32.412648 (+ 1095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:32.413063 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:32.413160 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:32.413963 (+ 803us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:15:32.413975 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:15:32.413975 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:15:32.414148 (+ 173us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:15:32.414182 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:15:32.414194 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:15:32.414250 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0807 15:15:32.414343 (+ 93us) server_negotiation.cc:299] Negotiation successful
0807 15:15:32.414426 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129,"thread_start_us":77,"threads_started":1}
I20260807 15:15:32.414721 25785 leader_election.cc:288] T 00000000000000000000000000000000 P 8d35796ccac14de3976172530653f7b9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 15:15:32.414803 25794 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8d35796ccac14de3976172530653f7b9 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:15:32.414997 25794 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8d35796ccac14de3976172530653f7b9 [term 1 LEADER]: Becoming Leader. State: Replica: 8d35796ccac14de3976172530653f7b9, State: Running, Role: LEADER
I20260807 15:15:32.415009 25785 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8d35796ccac14de3976172530653f7b9 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:15:32.415170 25794 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8d35796ccac14de3976172530653f7b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8d35796ccac14de3976172530653f7b9" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 46753 } }
I20260807 15:15:32.415530 25797 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8d35796ccac14de3976172530653f7b9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8d35796ccac14de3976172530653f7b9. Latest consensus state: current_term: 1 leader_uuid: "8d35796ccac14de3976172530653f7b9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8d35796ccac14de3976172530653f7b9" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 46753 } } }
I20260807 15:15:32.415541 25795 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8d35796ccac14de3976172530653f7b9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8d35796ccac14de3976172530653f7b9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8d35796ccac14de3976172530653f7b9" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 46753 } } }
I20260807 15:15:32.415642 25795 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8d35796ccac14de3976172530653f7b9 [sys.catalog]: This master's current role is: LEADER
I20260807 15:15:32.415797 25797 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8d35796ccac14de3976172530653f7b9 [sys.catalog]: This master's current role is: LEADER
I20260807 15:15:32.415921 25806 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:15:32.416604 25806 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:15:32.421612 25806 catalog_manager.cc:1347] Generated new cluster ID: e4deeace729f4df59aac63a251fdbef7
I20260807 15:15:32.421681 25806 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:15:32.457540 25806 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:15:32.458314 25806 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:15:32.476891 25806 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8d35796ccac14de3976172530653f7b9: Generated new TSK 0
I20260807 15:15:32.477130 25806 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 15:15:32.484524 30472 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7fZSIL/build/release/bin/kudu
/tmp/dist-test-task7fZSIL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115480851507-30472-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115480851507-30472-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115480851507-30472-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115480851507-30472-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.194.1:0
--local_ip_for_outbound_sockets=127.29.194.1
--webserver_interface=127.29.194.1
--webserver_port=0
--tserver_master_addrs=127.29.194.62:46753
--builtin_ntp_servers=127.29.194.20:37653
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260807 15:15:32.568881 25821 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:15:32.569105 25821 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:15:32.569136 25821 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:15:32.570343 25821 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:15:32.570386 25821 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:15:32.570429 25821 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.194.1
I20260807 15:15:32.571909 25821 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.194.20:37653
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115480851507-30472-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115480851507-30472-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.194.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115480851507-30472-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.194.1
--webserver_port=0
--tserver_master_addrs=127.29.194.62:46753
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25821
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.194.1
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115480851507-30472-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:15:32.572166 25821 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:15:32.572474 25821 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:15:32.577653 25830 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:15:32.577618 25821 server_base.cc:1034] running on GCE node
W20260807 15:15:32.577625 25829 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:15:32.577618 25833 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:15:32.578019 25821 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:15:32.578265 25821 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:15:32.579407 25821 hybrid_clock.cc:648] HybridClock initialized: now 1786115732579393 us; error 33 us; skew 500 ppm
I20260807 15:15:32.580641 25821 webserver.cc:460] Webserver started at http://127.29.194.1:40739/ using document root <none> and password file <none>
I20260807 15:15:32.580868 25821 fs_manager.cc:359] Metadata directory not provided
I20260807 15:15:32.580921 25821 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:15:32.581048 25821 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:15:32.582150 25821 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115480851507-30472-0/minicluster-data/ts-0/data/instance:
uuid: "a09691f64db94f8d80a439640268f442"
format_stamp: "Formatted at 2026-08-07 15:15:32 on dist-test-slave-42z9"
server_key: "e0c96fd0f7ae5966a21551ec0e3e72fb"
server_key_iv: "6756e68bbbc7cf9e0192c29228f6d5e7"
server_key_version: "encryptionkey@0"
I20260807 15:15:32.582515 25821 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115480851507-30472-0/minicluster-data/ts-0/wal/instance:
uuid: "a09691f64db94f8d80a439640268f442"
format_stamp: "Formatted at 2026-08-07 15:15:32 on dist-test-slave-42z9"
server_key: "e0c96fd0f7ae5966a21551ec0e3e72fb"
server_key_iv: "6756e68bbbc7cf9e0192c29228f6d5e7"
server_key_version: "encryptionkey@0"
I20260807 15:15:32.583781 25821 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 15:15:32.584647 25841 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:15:32.584836 25821 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:15:32.584932 25821 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115480851507-30472-0/minicluster-data/ts-0/data,/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115480851507-30472-0/minicluster-data/ts-0/wal
uuid: "a09691f64db94f8d80a439640268f442"
format_stamp: "Formatted at 2026-08-07 15:15:32 on dist-test-slave-42z9"
server_key: "e0c96fd0f7ae5966a21551ec0e3e72fb"
server_key_iv: "6756e68bbbc7cf9e0192c29228f6d5e7"
server_key_version: "encryptionkey@0"
I20260807 15:15:32.585002 25821 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115480851507-30472-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115480851507-30472-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115480851507-30472-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:15:32.600625 25821 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:15:32.600958 25821 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:15:32.601096 25821 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:15:32.601337 25821 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:15:32.601887 25821 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:15:32.601929 25821 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:15:32.601961 25821 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:15:32.601979 25821 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:15:32.611217 25821 rpc_server.cc:307] RPC server started. Bound to: 127.29.194.1:33343
I20260807 15:15:32.611249 26012 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.194.1:33343 every 8 connection(s)
I20260807 15:15:32.611807 25821 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115480851507-30472-0/minicluster-data/ts-0/data/info.pb
I20260807 15:15:32.618212 25792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:32.612620 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.194.1:43143 (local address 127.29.194.62:46753)
0807 15:15:32.612708 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:32.612711 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:32.613020 (+ 309us) server_negotiation.cc:407] Connection header received
0807 15:15:32.613744 (+ 724us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:32.613745 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:32.613758 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:32.613836 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:15:32.614798 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:32.615573 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:32.616719 (+ 1146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:32.616957 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:32.617043 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:32.617936 (+ 893us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:15:32.617937 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:15:32.617938 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:15:32.617943 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:15:32.617961 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:15:32.617966 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:15:32.618039 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0807 15:15:32.618089 (+ 50us) server_negotiation.cc:299] Negotiation successful
0807 15:15:32.618121 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260807 15:15:32.618383 26016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:32.612612 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:46753 (local address 127.29.194.1:43143)
0807 15:15:32.612864 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:32.612913 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0807 15:15:32.613631 (+ 718us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:15:32.613916 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:15:32.613921 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:15:32.613931 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:15:32.614680 (+ 749us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:32.614688 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:32.615668 (+ 980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:32.615669 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:32.616599 (+ 930us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:32.616602 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:32.617052 (+ 450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:32.617053 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:32.617212 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:32.617791 (+ 579us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 15:15:32.617791 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:15:32.617821 (+ 30us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 15:15:32.617847 (+ 26us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:15:32.618031 (+ 184us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:15:32.618036 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:15:32.618039 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 15:15:32.618087 (+ 48us) client_negotiation.cc:240] Negotiation successful
0807 15:15:32.618175 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":102,"threads_started":1}
I20260807 15:15:32.618893 26014 heartbeater.cc:347] Connected to a master server at 127.29.194.62:46753
I20260807 15:15:32.618996 26014 heartbeater.cc:464] Registering TS with master...
I20260807 15:15:32.619189 26014 heartbeater.cc:511] Master 127.29.194.62:46753 requested a full tablet report, sending...
I20260807 15:15:32.619764 25734 ts_manager.cc:194] Registered new tserver with Master: a09691f64db94f8d80a439640268f442 (127.29.194.1:33343)
I20260807 15:15:32.620522 25734 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.29.194.1:43143
I20260807 15:15:32.620950 30472 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7fZSIL/build/release/bin/kudu as pid 25821
I20260807 15:15:32.621078 30472 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115480851507-30472-0/minicluster-data/ts-0/wal/instance
I20260807 15:15:32.621305 30472 external_mini_cluster.cc:1428] Setting key cae345fadd84734c883f7bc6241458d1
I20260807 15:15:32.622254 30472 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7fZSIL/build/release/bin/kudu
/tmp/dist-test-task7fZSIL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115480851507-30472-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115480851507-30472-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115480851507-30472-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115480851507-30472-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.194.2:0
--local_ip_for_outbound_sockets=127.29.194.2
--webserver_interface=127.29.194.2
--webserver_port=0
--tserver_master_addrs=127.29.194.62:46753
--builtin_ntp_servers=127.29.194.20:37653
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260807 15:15:32.710013 26017 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:15:32.710230 26017 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:15:32.710253 26017 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:15:32.711441 26017 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:15:32.711488 26017 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:15:32.711514 26017 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.194.2
I20260807 15:15:32.712917 26017 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.194.20:37653
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115480851507-30472-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115480851507-30472-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.194.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115480851507-30472-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.194.2
--webserver_port=0
--tserver_master_addrs=127.29.194.62:46753
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26017
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.194.2
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115480851507-30472-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:15:32.713163 26017 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:15:32.713503 26017 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:15:32.718804 26026 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:15:32.718822 26028 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:15:32.718866 26017 server_base.cc:1034] running on GCE node
W20260807 15:15:32.718825 26025 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:15:32.719198 26017 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:15:32.719439 26017 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:15:32.720613 26017 hybrid_clock.cc:648] HybridClock initialized: now 1786115732720558 us; error 69 us; skew 500 ppm
I20260807 15:15:32.721949 26017 webserver.cc:460] Webserver started at http://127.29.194.2:33407/ using document root <none> and password file <none>
I20260807 15:15:32.722174 26017 fs_manager.cc:359] Metadata directory not provided
I20260807 15:15:32.722224 26017 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:15:32.722358 26017 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:15:32.723438 26017 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115480851507-30472-0/minicluster-data/ts-1/data/instance:
uuid: "667d880eb4cb4bec8737914e2b398f96"
format_stamp: "Formatted at 2026-08-07 15:15:32 on dist-test-slave-42z9"
server_key: "d512070ce1418d4cc15482ea1c793a30"
server_key_iv: "882a1854e13df082796584a99ed48d4c"
server_key_version: "encryptionkey@0"
I20260807 15:15:32.723773 26017 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115480851507-30472-0/minicluster-data/ts-1/wal/instance:
uuid: "667d880eb4cb4bec8737914e2b398f96"
format_stamp: "Formatted at 2026-08-07 15:15:32 on dist-test-slave-42z9"
server_key: "d512070ce1418d4cc15482ea1c793a30"
server_key_iv: "882a1854e13df082796584a99ed48d4c"
server_key_version: "encryptionkey@0"
I20260807 15:15:32.725029 26017 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 15:15:32.725765 26040 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:15:32.725905 26017 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260807 15:15:32.725991 26017 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115480851507-30472-0/minicluster-data/ts-1/data,/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115480851507-30472-0/minicluster-data/ts-1/wal
uuid: "667d880eb4cb4bec8737914e2b398f96"
format_stamp: "Formatted at 2026-08-07 15:15:32 on dist-test-slave-42z9"
server_key: "d512070ce1418d4cc15482ea1c793a30"
server_key_iv: "882a1854e13df082796584a99ed48d4c"
server_key_version: "encryptionkey@0"
I20260807 15:15:32.726051 26017 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115480851507-30472-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115480851507-30472-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115480851507-30472-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:15:32.741114 26017 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:15:32.741472 26017 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:15:32.741624 26017 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:15:32.741865 26017 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:15:32.742202 26017 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:15:32.742239 26017 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:15:32.742269 26017 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:15:32.742291 26017 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:15:32.747852 26017 rpc_server.cc:307] RPC server started. Bound to: 127.29.194.2:46075
I20260807 15:15:32.747898 26204 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.194.2:46075 every 8 connection(s)
I20260807 15:15:32.748234 26017 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115480851507-30472-0/minicluster-data/ts-1/data/info.pb
I20260807 15:15:32.754503 25792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:32.749296 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.194.2:34373 (local address 127.29.194.62:46753)
0807 15:15:32.749375 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:32.749377 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:32.749610 (+ 233us) server_negotiation.cc:407] Connection header received
0807 15:15:32.750124 (+ 514us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:32.750125 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:32.750136 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:32.750199 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:15:32.750932 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:32.751697 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:32.752728 (+ 1031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:32.752921 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:32.752983 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:32.754045 (+ 1062us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:15:32.754046 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:15:32.754047 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:15:32.754051 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:15:32.754066 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:15:32.754070 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:15:32.754127 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0807 15:15:32.754398 (+ 271us) server_negotiation.cc:299] Negotiation successful
0807 15:15:32.754438 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260807 15:15:32.754894 26210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:32.749227 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:46753 (local address 127.29.194.2:34373)
0807 15:15:32.749472 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:32.749516 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0807 15:15:32.750011 (+ 495us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:15:32.750264 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:15:32.750267 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:15:32.750276 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:15:32.750814 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:32.750819 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:32.751812 (+ 993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:32.751813 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:32.752629 (+ 816us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:32.752632 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:32.752991 (+ 359us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:32.752992 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:32.753099 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:32.753873 (+ 774us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 15:15:32.753874 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:15:32.753910 (+ 36us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 15:15:32.753934 (+ 24us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:15:32.754259 (+ 325us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:15:32.754273 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:15:32.754278 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 15:15:32.754399 (+ 121us) client_negotiation.cc:240] Negotiation successful
0807 15:15:32.754549 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":108,"threads_started":1}
I20260807 15:15:32.755486 26205 heartbeater.cc:347] Connected to a master server at 127.29.194.62:46753
I20260807 15:15:32.755611 26205 heartbeater.cc:464] Registering TS with master...
I20260807 15:15:32.755805 26205 heartbeater.cc:511] Master 127.29.194.62:46753 requested a full tablet report, sending...
I20260807 15:15:32.756276 25734 ts_manager.cc:194] Registered new tserver with Master: 667d880eb4cb4bec8737914e2b398f96 (127.29.194.2:46075)
I20260807 15:15:32.756640 25734 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.29.194.2:34373
I20260807 15:15:32.758066 30472 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7fZSIL/build/release/bin/kudu as pid 26017
I20260807 15:15:32.758172 30472 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115480851507-30472-0/minicluster-data/ts-1/wal/instance
I20260807 15:15:32.758358 30472 external_mini_cluster.cc:1428] Setting key ff382d26cb6ba766eb7ea8c03653101a
I20260807 15:15:32.759248 30472 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7fZSIL/build/release/bin/kudu
/tmp/dist-test-task7fZSIL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115480851507-30472-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115480851507-30472-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115480851507-30472-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115480851507-30472-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.194.3:0
--local_ip_for_outbound_sockets=127.29.194.3
--webserver_interface=127.29.194.3
--webserver_port=0
--tserver_master_addrs=127.29.194.62:46753
--builtin_ntp_servers=127.29.194.20:37653
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260807 15:15:32.843170 26211 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:15:32.843407 26211 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:15:32.843430 26211 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:15:32.844606 26211 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:15:32.844652 26211 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:15:32.844677 26211 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.194.3
I20260807 15:15:32.846104 26211 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.194.20:37653
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115480851507-30472-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115480851507-30472-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.194.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115480851507-30472-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.194.3
--webserver_port=0
--tserver_master_addrs=127.29.194.62:46753
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26211
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.194.3
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115480851507-30472-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:15:32.846354 26211 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:15:32.846657 26211 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:15:32.851668 26226 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:15:32.851740 26222 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:15:32.851863 26221 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:15:32.852037 26211 server_base.cc:1034] running on GCE node
I20260807 15:15:32.852246 26211 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:15:32.852485 26211 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:15:32.853663 26211 hybrid_clock.cc:648] HybridClock initialized: now 1786115732853647 us; error 31 us; skew 500 ppm
I20260807 15:15:32.855031 26211 webserver.cc:460] Webserver started at http://127.29.194.3:36117/ using document root <none> and password file <none>
I20260807 15:15:32.855265 26211 fs_manager.cc:359] Metadata directory not provided
I20260807 15:15:32.855422 26211 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:15:32.855582 26211 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:15:32.856671 26211 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115480851507-30472-0/minicluster-data/ts-2/data/instance:
uuid: "0c61eae055234692acb4b7dc0250a012"
format_stamp: "Formatted at 2026-08-07 15:15:32 on dist-test-slave-42z9"
server_key: "20f7bfe63368a20b98fc84a454d71ee5"
server_key_iv: "abad38f55b77a060c56da6a68b6599a6"
server_key_version: "encryptionkey@0"
I20260807 15:15:32.857012 26211 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115480851507-30472-0/minicluster-data/ts-2/wal/instance:
uuid: "0c61eae055234692acb4b7dc0250a012"
format_stamp: "Formatted at 2026-08-07 15:15:32 on dist-test-slave-42z9"
server_key: "20f7bfe63368a20b98fc84a454d71ee5"
server_key_iv: "abad38f55b77a060c56da6a68b6599a6"
server_key_version: "encryptionkey@0"
I20260807 15:15:32.858315 26211 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260807 15:15:32.859005 26238 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:15:32.859174 26211 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260807 15:15:32.859269 26211 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115480851507-30472-0/minicluster-data/ts-2/data,/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115480851507-30472-0/minicluster-data/ts-2/wal
uuid: "0c61eae055234692acb4b7dc0250a012"
format_stamp: "Formatted at 2026-08-07 15:15:32 on dist-test-slave-42z9"
server_key: "20f7bfe63368a20b98fc84a454d71ee5"
server_key_iv: "abad38f55b77a060c56da6a68b6599a6"
server_key_version: "encryptionkey@0"
I20260807 15:15:32.859329 26211 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115480851507-30472-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115480851507-30472-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115480851507-30472-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:15:32.887929 26211 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:15:32.888283 26211 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:15:32.888432 26211 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:15:32.888684 26211 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:15:32.889060 26211 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:15:32.889111 26211 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:15:32.889134 26211 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:15:32.889146 26211 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:15:32.894915 26211 rpc_server.cc:307] RPC server started. Bound to: 127.29.194.3:35727
I20260807 15:15:32.894966 26400 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.194.3:35727 every 8 connection(s)
I20260807 15:15:32.895308 26211 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115480851507-30472-0/minicluster-data/ts-2/data/info.pb
I20260807 15:15:32.895690 30472 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7fZSIL/build/release/bin/kudu as pid 26211
I20260807 15:15:32.895807 30472 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115480851507-30472-0/minicluster-data/ts-2/wal/instance
I20260807 15:15:32.896005 30472 external_mini_cluster.cc:1428] Setting key 0add95cc19428821b2d6ae8e7efd34cf
I20260807 15:15:32.900954 25792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:32.896110 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.194.3:48375 (local address 127.29.194.62:46753)
0807 15:15:32.896172 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:32.896174 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:32.896391 (+ 217us) server_negotiation.cc:407] Connection header received
0807 15:15:32.897037 (+ 646us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:32.897037 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:32.897046 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:32.897079 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:15:32.897804 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:32.898312 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:32.899417 (+ 1105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:32.899679 (+ 262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:32.899790 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:32.900656 (+ 866us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:15:32.900657 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:15:32.900658 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:15:32.900662 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:15:32.900684 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:15:32.900689 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:15:32.900736 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0807 15:15:32.900858 (+ 122us) server_negotiation.cc:299] Negotiation successful
0807 15:15:32.900889 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260807 15:15:32.901192 26406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:32.896066 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:46753 (local address 127.29.194.3:48375)
0807 15:15:32.896274 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:32.896321 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0807 15:15:32.896963 (+ 642us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:15:32.897135 (+ 172us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:15:32.897138 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:15:32.897146 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:15:32.897727 (+ 581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:32.897732 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:32.898415 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:32.898416 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:32.899247 (+ 831us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:32.899249 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:32.899828 (+ 579us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:32.899829 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:32.899961 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:32.900495 (+ 534us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 15:15:32.900496 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:15:32.900526 (+ 30us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 15:15:32.900551 (+ 25us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:15:32.900773 (+ 222us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:15:32.900780 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:15:32.900783 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 15:15:32.900857 (+ 74us) client_negotiation.cc:240] Negotiation successful
0807 15:15:32.900948 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":86,"threads_started":1}
I20260807 15:15:32.901710 26402 heartbeater.cc:347] Connected to a master server at 127.29.194.62:46753
I20260807 15:15:32.901799 26402 heartbeater.cc:464] Registering TS with master...
I20260807 15:15:32.901980 26402 heartbeater.cc:511] Master 127.29.194.62:46753 requested a full tablet report, sending...
I20260807 15:15:32.902333 25734 ts_manager.cc:194] Registered new tserver with Master: 0c61eae055234692acb4b7dc0250a012 (127.29.194.3:35727)
I20260807 15:15:32.902693 25734 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.29.194.3:48375
I20260807 15:15:32.908027 30472 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 15:15:32.912529 25792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:32.909525 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48890 (local address 127.29.194.62:46753)
0807 15:15:32.909567 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:32.909569 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:32.909640 (+ 71us) server_negotiation.cc:407] Connection header received
0807 15:15:32.909694 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:32.909695 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:32.909702 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:32.909750 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:15:32.909885 (+ 135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:32.910368 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:32.911195 (+ 827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:32.911391 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:32.911446 (+ 55us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:32.912189 (+ 743us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:15:32.912190 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:15:32.912190 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:15:32.912195 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:15:32.912208 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:15:32.912212 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:15:32.912281 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0807 15:15:32.912433 (+ 152us) server_negotiation.cc:299] Negotiation successful
0807 15:15:32.912446 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260807 15:15:32.914328 25734 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48890:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 15:15:32.916023 25734 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 15:15:32.923718 26412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:32.918255 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.1:33343 (local address 127.0.0.1:58436)
0807 15:15:32.918419 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:32.918449 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0807 15:15:32.918516 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:15:32.918889 (+ 373us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:15:32.918892 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:15:32.918894 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:15:32.918980 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:32.918982 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:32.920464 (+ 1482us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:32.920467 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:32.922345 (+ 1878us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:32.922349 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:32.923394 (+ 1045us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:32.923395 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:32.923469 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:32.923475 (+ 6us) client_negotiation.cc:769] Sending connection context
0807 15:15:32.923517 (+ 42us) client_negotiation.cc:240] Negotiation successful
0807 15:15:32.923585 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"mutex_wait_us":86,"thread_start_us":62,"threads_started":1}
I20260807 15:15:32.923848 26416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:32.918262 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.3:35727 (local address 127.0.0.1:46394)
0807 15:15:32.918690 (+ 428us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:32.918701 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 15:15:32.918756 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:15:32.919196 (+ 440us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:15:32.919197 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:15:32.919198 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:15:32.919235 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:32.919236 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:32.921367 (+ 2131us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:32.921369 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:32.922257 (+ 888us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:32.922260 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:32.923236 (+ 976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:32.923238 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:32.923333 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:32.923356 (+ 23us) client_negotiation.cc:769] Sending connection context
0807 15:15:32.923442 (+ 86us) client_negotiation.cc:240] Negotiation successful
0807 15:15:32.923522 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":415,"thread_start_us":57,"threads_started":1}
I20260807 15:15:32.923866 26419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:32.918702 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46394 (local address 127.29.194.3:35727)
0807 15:15:32.918849 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:32.918860 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:32.918871 (+ 11us) server_negotiation.cc:407] Connection header received
0807 15:15:32.919059 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:32.919061 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:32.919075 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:32.919140 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:15:32.920451 (+ 1311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:32.921295 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:32.922342 (+ 1047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:32.923090 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:32.923176 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:32.923283 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0807 15:15:32.923725 (+ 442us) server_negotiation.cc:299] Negotiation successful
0807 15:15:32.923770 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":114,"thread_start_us":75,"threads_started":1}
I20260807 15:15:32.923945 26418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:32.918517 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58436 (local address 127.29.194.1:33343)
0807 15:15:32.918694 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:32.918699 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:32.918718 (+ 19us) server_negotiation.cc:407] Connection header received
0807 15:15:32.918761 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:32.918763 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:32.918779 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:32.919056 (+ 277us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:15:32.919118 (+ 62us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:32.920278 (+ 1160us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:32.922437 (+ 2159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:32.923121 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:32.923689 (+ 568us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:32.923795 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0807 15:15:32.923823 (+ 28us) server_negotiation.cc:299] Negotiation successful
0807 15:15:32.923866 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":139,"thread_start_us":92,"threads_started":1}
I20260807 15:15:32.923970 26417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:32.918511 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54432 (local address 127.29.194.2:46075)
0807 15:15:32.918672 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:32.918676 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:32.918876 (+ 200us) server_negotiation.cc:407] Connection header received
0807 15:15:32.918980 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:32.918983 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:32.918998 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:32.919074 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:15:32.919284 (+ 210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:32.920356 (+ 1072us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:32.922494 (+ 2138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:32.923248 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:32.923365 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:32.923472 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0807 15:15:32.923800 (+ 328us) server_negotiation.cc:299] Negotiation successful
0807 15:15:32.923871 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":95,"threads_started":1}
I20260807 15:15:32.924580 26415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:32.918262 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.2:46075 (local address 127.0.0.1:54432)
0807 15:15:32.918820 (+ 558us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:32.918828 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0807 15:15:32.918901 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:15:32.919141 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:15:32.919142 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:15:32.919144 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:15:32.919188 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:32.919190 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:32.920466 (+ 1276us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:32.920467 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:32.922345 (+ 1878us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:32.922349 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:32.923510 (+ 1161us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:32.923511 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:32.923565 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:32.923570 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 15:15:32.924436 (+ 866us) client_negotiation.cc:240] Negotiation successful
0807 15:15:32.924509 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":550,"mutex_wait_us":92,"thread_start_us":54,"threads_started":1}
I20260807 15:15:32.924633 26305 tablet_service.cc:1467] Processing CreateTablet for tablet b12cd0a6ba3c4d73ae671b6b81f71c3f (DEFAULT_TABLE table=test-table [id=ed2b49a247bb426082584fff22677234]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:15:32.924950 25914 tablet_service.cc:1467] Processing CreateTablet for tablet b12cd0a6ba3c4d73ae671b6b81f71c3f (DEFAULT_TABLE table=test-table [id=ed2b49a247bb426082584fff22677234]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:15:32.925062 26305 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b12cd0a6ba3c4d73ae671b6b81f71c3f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:15:32.925271 25914 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b12cd0a6ba3c4d73ae671b6b81f71c3f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:15:32.925983 26109 tablet_service.cc:1467] Processing CreateTablet for tablet b12cd0a6ba3c4d73ae671b6b81f71c3f (DEFAULT_TABLE table=test-table [id=ed2b49a247bb426082584fff22677234]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:15:32.926492 26109 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b12cd0a6ba3c4d73ae671b6b81f71c3f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:15:32.929343 26424 tablet_bootstrap.cc:492] T b12cd0a6ba3c4d73ae671b6b81f71c3f P 667d880eb4cb4bec8737914e2b398f96: Bootstrap starting.
I20260807 15:15:32.929611 26422 tablet_bootstrap.cc:492] T b12cd0a6ba3c4d73ae671b6b81f71c3f P 0c61eae055234692acb4b7dc0250a012: Bootstrap starting.
I20260807 15:15:32.930116 26424 tablet_bootstrap.cc:654] T b12cd0a6ba3c4d73ae671b6b81f71c3f P 667d880eb4cb4bec8737914e2b398f96: Neither blocks nor log segments found. Creating new log.
I20260807 15:15:32.930271 26422 tablet_bootstrap.cc:654] T b12cd0a6ba3c4d73ae671b6b81f71c3f P 0c61eae055234692acb4b7dc0250a012: Neither blocks nor log segments found. Creating new log.
I20260807 15:15:32.930405 26424 log.cc:824] T b12cd0a6ba3c4d73ae671b6b81f71c3f P 667d880eb4cb4bec8737914e2b398f96: Log is configured to *not* fsync() on all Append() calls
I20260807 15:15:32.930668 26422 log.cc:824] T b12cd0a6ba3c4d73ae671b6b81f71c3f P 0c61eae055234692acb4b7dc0250a012: Log is configured to *not* fsync() on all Append() calls
I20260807 15:15:32.930950 26420 tablet_bootstrap.cc:492] T b12cd0a6ba3c4d73ae671b6b81f71c3f P a09691f64db94f8d80a439640268f442: Bootstrap starting.
I20260807 15:15:32.931164 26424 tablet_bootstrap.cc:492] T b12cd0a6ba3c4d73ae671b6b81f71c3f P 667d880eb4cb4bec8737914e2b398f96: No bootstrap required, opened a new log
I20260807 15:15:32.931234 26424 ts_tablet_manager.cc:1397] T b12cd0a6ba3c4d73ae671b6b81f71c3f P 667d880eb4cb4bec8737914e2b398f96: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 15:15:32.931354 26422 tablet_bootstrap.cc:492] T b12cd0a6ba3c4d73ae671b6b81f71c3f P 0c61eae055234692acb4b7dc0250a012: No bootstrap required, opened a new log
I20260807 15:15:32.931417 26422 ts_tablet_manager.cc:1397] T b12cd0a6ba3c4d73ae671b6b81f71c3f P 0c61eae055234692acb4b7dc0250a012: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 15:15:32.931609 26420 tablet_bootstrap.cc:654] T b12cd0a6ba3c4d73ae671b6b81f71c3f P a09691f64db94f8d80a439640268f442: Neither blocks nor log segments found. Creating new log.
I20260807 15:15:32.931865 26420 log.cc:824] T b12cd0a6ba3c4d73ae671b6b81f71c3f P a09691f64db94f8d80a439640268f442: Log is configured to *not* fsync() on all Append() calls
I20260807 15:15:32.932498 26420 tablet_bootstrap.cc:492] T b12cd0a6ba3c4d73ae671b6b81f71c3f P a09691f64db94f8d80a439640268f442: No bootstrap required, opened a new log
I20260807 15:15:32.932571 26420 ts_tablet_manager.cc:1397] T b12cd0a6ba3c4d73ae671b6b81f71c3f P a09691f64db94f8d80a439640268f442: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 15:15:32.933837 26424 raft_consensus.cc:348] T b12cd0a6ba3c4d73ae671b6b81f71c3f P 667d880eb4cb4bec8737914e2b398f96 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a09691f64db94f8d80a439640268f442" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 33343 } } peers { permanent_uuid: "667d880eb4cb4bec8737914e2b398f96" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 46075 } } peers { permanent_uuid: "0c61eae055234692acb4b7dc0250a012" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 35727 } }
I20260807 15:15:32.934024 26424 raft_consensus.cc:374] T b12cd0a6ba3c4d73ae671b6b81f71c3f P 667d880eb4cb4bec8737914e2b398f96 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:15:32.934065 26424 raft_consensus.cc:729] T b12cd0a6ba3c4d73ae671b6b81f71c3f P 667d880eb4cb4bec8737914e2b398f96 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 667d880eb4cb4bec8737914e2b398f96, State: Initialized, Role: FOLLOWER
I20260807 15:15:32.934211 26424 consensus_queue.cc:260] T b12cd0a6ba3c4d73ae671b6b81f71c3f P 667d880eb4cb4bec8737914e2b398f96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a09691f64db94f8d80a439640268f442" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 33343 } } peers { permanent_uuid: "667d880eb4cb4bec8737914e2b398f96" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 46075 } } peers { permanent_uuid: "0c61eae055234692acb4b7dc0250a012" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 35727 } }
I20260807 15:15:32.934149 26422 raft_consensus.cc:348] T b12cd0a6ba3c4d73ae671b6b81f71c3f P 0c61eae055234692acb4b7dc0250a012 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a09691f64db94f8d80a439640268f442" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 33343 } } peers { permanent_uuid: "667d880eb4cb4bec8737914e2b398f96" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 46075 } } peers { permanent_uuid: "0c61eae055234692acb4b7dc0250a012" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 35727 } }
I20260807 15:15:32.934283 26422 raft_consensus.cc:374] T b12cd0a6ba3c4d73ae671b6b81f71c3f P 0c61eae055234692acb4b7dc0250a012 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:15:32.934314 26422 raft_consensus.cc:729] T b12cd0a6ba3c4d73ae671b6b81f71c3f P 0c61eae055234692acb4b7dc0250a012 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0c61eae055234692acb4b7dc0250a012, State: Initialized, Role: FOLLOWER
I20260807 15:15:32.934445 26422 consensus_queue.cc:260] T b12cd0a6ba3c4d73ae671b6b81f71c3f P 0c61eae055234692acb4b7dc0250a012 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a09691f64db94f8d80a439640268f442" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 33343 } } peers { permanent_uuid: "667d880eb4cb4bec8737914e2b398f96" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 46075 } } peers { permanent_uuid: "0c61eae055234692acb4b7dc0250a012" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 35727 } }
I20260807 15:15:32.934485 26424 ts_tablet_manager.cc:1428] T b12cd0a6ba3c4d73ae671b6b81f71c3f P 667d880eb4cb4bec8737914e2b398f96: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260807 15:15:32.934613 26205 heartbeater.cc:503] Master 127.29.194.62:46753 was elected leader, sending a full tablet report...
I20260807 15:15:32.934687 26422 ts_tablet_manager.cc:1428] T b12cd0a6ba3c4d73ae671b6b81f71c3f P 0c61eae055234692acb4b7dc0250a012: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260807 15:15:32.934599 26420 raft_consensus.cc:348] T b12cd0a6ba3c4d73ae671b6b81f71c3f P a09691f64db94f8d80a439640268f442 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a09691f64db94f8d80a439640268f442" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 33343 } } peers { permanent_uuid: "667d880eb4cb4bec8737914e2b398f96" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 46075 } } peers { permanent_uuid: "0c61eae055234692acb4b7dc0250a012" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 35727 } }
I20260807 15:15:32.934763 26420 raft_consensus.cc:374] T b12cd0a6ba3c4d73ae671b6b81f71c3f P a09691f64db94f8d80a439640268f442 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:15:32.934790 26402 heartbeater.cc:503] Master 127.29.194.62:46753 was elected leader, sending a full tablet report...
I20260807 15:15:32.934803 26420 raft_consensus.cc:729] T b12cd0a6ba3c4d73ae671b6b81f71c3f P a09691f64db94f8d80a439640268f442 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a09691f64db94f8d80a439640268f442, State: Initialized, Role: FOLLOWER
I20260807 15:15:32.934906 26420 consensus_queue.cc:260] T b12cd0a6ba3c4d73ae671b6b81f71c3f P a09691f64db94f8d80a439640268f442 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a09691f64db94f8d80a439640268f442" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 33343 } } peers { permanent_uuid: "667d880eb4cb4bec8737914e2b398f96" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 46075 } } peers { permanent_uuid: "0c61eae055234692acb4b7dc0250a012" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 35727 } }
I20260807 15:15:32.935146 26420 ts_tablet_manager.cc:1428] T b12cd0a6ba3c4d73ae671b6b81f71c3f P a09691f64db94f8d80a439640268f442: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260807 15:15:32.935256 26014 heartbeater.cc:503] Master 127.29.194.62:46753 was elected leader, sending a full tablet report...
I20260807 15:15:32.958933 26433 raft_consensus.cc:482] T b12cd0a6ba3c4d73ae671b6b81f71c3f P 0c61eae055234692acb4b7dc0250a012 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 15:15:32.959095 26433 raft_consensus.cc:504] T b12cd0a6ba3c4d73ae671b6b81f71c3f P 0c61eae055234692acb4b7dc0250a012 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a09691f64db94f8d80a439640268f442" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 33343 } } peers { permanent_uuid: "667d880eb4cb4bec8737914e2b398f96" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 46075 } } peers { permanent_uuid: "0c61eae055234692acb4b7dc0250a012" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 35727 } }
I20260807 15:15:32.959491 26433 leader_election.cc:288] T b12cd0a6ba3c4d73ae671b6b81f71c3f P 0c61eae055234692acb4b7dc0250a012 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a09691f64db94f8d80a439640268f442 (127.29.194.1:33343), 667d880eb4cb4bec8737914e2b398f96 (127.29.194.2:46075)
I20260807 15:15:32.962602 26417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:32.959954 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.194.3:49787 (local address 127.29.194.2:46075)
0807 15:15:32.959999 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:32.960000 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:32.960011 (+ 11us) server_negotiation.cc:407] Connection header received
0807 15:15:32.960040 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:32.960041 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:32.960051 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:32.960140 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:15:32.960330 (+ 190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:32.960810 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:32.961874 (+ 1064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:32.962277 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:32.962345 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:32.962355 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0807 15:15:32.962490 (+ 135us) server_negotiation.cc:299] Negotiation successful
0807 15:15:32.962530 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260807 15:15:32.962612 26406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:32.959816 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.2:46075 (local address 127.29.194.3:49787)
0807 15:15:32.959905 (+ 89us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:32.959919 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 15:15:32.959974 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:15:32.960173 (+ 199us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:15:32.960175 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:15:32.960178 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:15:32.960255 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:32.960257 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:32.960894 (+ 637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:32.960896 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:32.961789 (+ 893us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:32.961791 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:32.962350 (+ 559us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:32.962351 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:32.962429 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:32.962433 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 15:15:32.962478 (+ 45us) client_negotiation.cc:240] Negotiation successful
0807 15:15:32.962528 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":58}
I20260807 15:15:32.963064 26418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:32.959891 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.194.3:51575 (local address 127.29.194.1:33343)
0807 15:15:32.959965 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:32.959967 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:32.960251 (+ 284us) server_negotiation.cc:407] Connection header received
0807 15:15:32.960299 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:32.960302 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:32.960318 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:32.960365 (+ 47us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:15:32.960536 (+ 171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:32.961047 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:32.962252 (+ 1205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:32.962677 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:32.962758 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:32.962766 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0807 15:15:32.962944 (+ 178us) server_negotiation.cc:299] Negotiation successful
0807 15:15:32.963000 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260807 15:15:32.963056 26143 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b12cd0a6ba3c4d73ae671b6b81f71c3f" candidate_uuid: "0c61eae055234692acb4b7dc0250a012" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "667d880eb4cb4bec8737914e2b398f96" is_pre_election: true
I20260807 15:15:32.963246 26143 raft_consensus.cc:2454] T b12cd0a6ba3c4d73ae671b6b81f71c3f P 667d880eb4cb4bec8737914e2b398f96 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0c61eae055234692acb4b7dc0250a012 in term 0.
I20260807 15:15:32.963325 26437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:32.959816 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.1:33343 (local address 127.29.194.3:51575)
0807 15:15:32.960167 (+ 351us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:32.960176 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0807 15:15:32.960234 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:15:32.960373 (+ 139us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:15:32.960374 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:15:32.960376 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:15:32.960473 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:32.960475 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:32.961123 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:32.961124 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:32.962136 (+ 1012us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:32.962140 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:32.962777 (+ 637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:32.962778 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:32.962851 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:32.962856 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 15:15:32.963191 (+ 335us) client_negotiation.cc:240] Negotiation successful
0807 15:15:32.963240 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":334,"mutex_wait_us":101,"thread_start_us":180,"threads_started":1}
I20260807 15:15:32.963488 26241 leader_election.cc:302] T b12cd0a6ba3c4d73ae671b6b81f71c3f P 0c61eae055234692acb4b7dc0250a012 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0c61eae055234692acb4b7dc0250a012, 667d880eb4cb4bec8737914e2b398f96; no voters:
I20260807 15:15:32.963639 26433 raft_consensus.cc:2781] T b12cd0a6ba3c4d73ae671b6b81f71c3f P 0c61eae055234692acb4b7dc0250a012 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 15:15:32.963670 25944 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b12cd0a6ba3c4d73ae671b6b81f71c3f" candidate_uuid: "0c61eae055234692acb4b7dc0250a012" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a09691f64db94f8d80a439640268f442" is_pre_election: true
I20260807 15:15:32.963732 26433 raft_consensus.cc:482] T b12cd0a6ba3c4d73ae671b6b81f71c3f P 0c61eae055234692acb4b7dc0250a012 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 15:15:32.963757 26433 raft_consensus.cc:3037] T b12cd0a6ba3c4d73ae671b6b81f71c3f P 0c61eae055234692acb4b7dc0250a012 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:15:32.963802 25944 raft_consensus.cc:2454] T b12cd0a6ba3c4d73ae671b6b81f71c3f P a09691f64db94f8d80a439640268f442 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0c61eae055234692acb4b7dc0250a012 in term 0.
I20260807 15:15:32.964519 26433 raft_consensus.cc:504] T b12cd0a6ba3c4d73ae671b6b81f71c3f P 0c61eae055234692acb4b7dc0250a012 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a09691f64db94f8d80a439640268f442" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 33343 } } peers { permanent_uuid: "667d880eb4cb4bec8737914e2b398f96" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 46075 } } peers { permanent_uuid: "0c61eae055234692acb4b7dc0250a012" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 35727 } }
I20260807 15:15:32.964672 26433 leader_election.cc:288] T b12cd0a6ba3c4d73ae671b6b81f71c3f P 0c61eae055234692acb4b7dc0250a012 [CANDIDATE]: Term 1 election: Requested vote from peers a09691f64db94f8d80a439640268f442 (127.29.194.1:33343), 667d880eb4cb4bec8737914e2b398f96 (127.29.194.2:46075)
I20260807 15:15:32.964828 26143 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b12cd0a6ba3c4d73ae671b6b81f71c3f" candidate_uuid: "0c61eae055234692acb4b7dc0250a012" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "667d880eb4cb4bec8737914e2b398f96"
I20260807 15:15:32.964839 25944 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b12cd0a6ba3c4d73ae671b6b81f71c3f" candidate_uuid: "0c61eae055234692acb4b7dc0250a012" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a09691f64db94f8d80a439640268f442"
I20260807 15:15:32.964918 25944 raft_consensus.cc:3037] T b12cd0a6ba3c4d73ae671b6b81f71c3f P a09691f64db94f8d80a439640268f442 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:15:32.964916 26143 raft_consensus.cc:3037] T b12cd0a6ba3c4d73ae671b6b81f71c3f P 667d880eb4cb4bec8737914e2b398f96 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:15:32.965900 25944 raft_consensus.cc:2454] T b12cd0a6ba3c4d73ae671b6b81f71c3f P a09691f64db94f8d80a439640268f442 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0c61eae055234692acb4b7dc0250a012 in term 1.
I20260807 15:15:32.966037 26143 raft_consensus.cc:2454] T b12cd0a6ba3c4d73ae671b6b81f71c3f P 667d880eb4cb4bec8737914e2b398f96 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0c61eae055234692acb4b7dc0250a012 in term 1.
I20260807 15:15:32.966109 26240 leader_election.cc:302] T b12cd0a6ba3c4d73ae671b6b81f71c3f P 0c61eae055234692acb4b7dc0250a012 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0c61eae055234692acb4b7dc0250a012, a09691f64db94f8d80a439640268f442; no voters:
I20260807 15:15:32.966212 26433 raft_consensus.cc:2781] T b12cd0a6ba3c4d73ae671b6b81f71c3f P 0c61eae055234692acb4b7dc0250a012 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:15:32.966382 26433 raft_consensus.cc:686] T b12cd0a6ba3c4d73ae671b6b81f71c3f P 0c61eae055234692acb4b7dc0250a012 [term 1 LEADER]: Becoming Leader. State: Replica: 0c61eae055234692acb4b7dc0250a012, State: Running, Role: LEADER
I20260807 15:15:32.966555 26433 consensus_queue.cc:237] T b12cd0a6ba3c4d73ae671b6b81f71c3f P 0c61eae055234692acb4b7dc0250a012 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a09691f64db94f8d80a439640268f442" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 33343 } } peers { permanent_uuid: "667d880eb4cb4bec8737914e2b398f96" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 46075 } } peers { permanent_uuid: "0c61eae055234692acb4b7dc0250a012" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 35727 } }
I20260807 15:15:32.967557 25733 catalog_manager.cc:5495] T b12cd0a6ba3c4d73ae671b6b81f71c3f P 0c61eae055234692acb4b7dc0250a012 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0c61eae055234692acb4b7dc0250a012 (127.29.194.3). New cstate: current_term: 1 leader_uuid: "0c61eae055234692acb4b7dc0250a012" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a09691f64db94f8d80a439640268f442" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 33343 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "667d880eb4cb4bec8737914e2b398f96" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 46075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0c61eae055234692acb4b7dc0250a012" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 35727 } health_report { overall_health: HEALTHY } } }
I20260807 15:15:32.982591 26419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:32.980150 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46404 (local address 127.29.194.3:35727)
0807 15:15:32.980218 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:32.980220 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:32.980229 (+ 9us) server_negotiation.cc:407] Connection header received
0807 15:15:32.980277 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:32.980278 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:32.980302 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:32.980360 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 15:15:32.980479 (+ 119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:32.980944 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:32.981881 (+ 937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:32.982073 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:32.982125 (+ 52us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:32.982265 (+ 140us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 15:15:32.982354 (+ 89us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 15:15:32.982439 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0807 15:15:32.982484 (+ 45us) server_negotiation.cc:299] Negotiation successful
0807 15:15:32.982494 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260807 15:15:32.984422 25944 raft_consensus.cc:1261] T b12cd0a6ba3c4d73ae671b6b81f71c3f P a09691f64db94f8d80a439640268f442 [term 1 FOLLOWER]: Refusing update from remote peer 0c61eae055234692acb4b7dc0250a012: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:15:32.984705 26143 raft_consensus.cc:1261] T b12cd0a6ba3c4d73ae671b6b81f71c3f P 667d880eb4cb4bec8737914e2b398f96 [term 1 FOLLOWER]: Refusing update from remote peer 0c61eae055234692acb4b7dc0250a012: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:15:32.984789 26438 consensus_queue.cc:1035] T b12cd0a6ba3c4d73ae671b6b81f71c3f P 0c61eae055234692acb4b7dc0250a012 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a09691f64db94f8d80a439640268f442" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 33343 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:15:32.984968 26438 consensus_queue.cc:1035] T b12cd0a6ba3c4d73ae671b6b81f71c3f P 0c61eae055234692acb4b7dc0250a012 [LEADER]: Connected to new peer: Peer: permanent_uuid: "667d880eb4cb4bec8737914e2b398f96" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 46075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:15:32.987385 26441 mvcc.cc:205] Tried to move back new op lower bound from 7315930042301607936 to 7315930042231832576. Current Snapshot: MvccSnapshot[applied={T|T < 7315930042301607936}]
I20260807 15:15:32.988067 26446 mvcc.cc:205] Tried to move back new op lower bound from 7315930042301607936 to 7315930042231832576. Current Snapshot: MvccSnapshot[applied={T|T < 7315930042301607936}]
I20260807 15:15:32.989814 25733 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:48890:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 15:15:32.989949 25733 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:48890:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 15:15:32.990703 25733 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8d35796ccac14de3976172530653f7b9: Sending DeleteTablet for 3 replicas of tablet b12cd0a6ba3c4d73ae671b6b81f71c3f
I20260807 15:15:32.991029 25914 tablet_service.cc:1514] Processing DeleteTablet for tablet b12cd0a6ba3c4d73ae671b6b81f71c3f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:15:32 UTC) from {username='slave'} at 127.0.0.1:58436
I20260807 15:15:32.991048 26109 tablet_service.cc:1514] Processing DeleteTablet for tablet b12cd0a6ba3c4d73ae671b6b81f71c3f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:15:32 UTC) from {username='slave'} at 127.0.0.1:54432
I20260807 15:15:32.991442 26459 tablet_replica.cc:330] T b12cd0a6ba3c4d73ae671b6b81f71c3f P a09691f64db94f8d80a439640268f442: stopping tablet replica
I20260807 15:15:32.991505 26305 tablet_service.cc:1514] Processing DeleteTablet for tablet b12cd0a6ba3c4d73ae671b6b81f71c3f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:15:32 UTC) from {username='slave'} at 127.0.0.1:46394
I20260807 15:15:32.991565 30472 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7fZSIL/build/release/bin/kudu with pid 25821
I20260807 15:15:32.991650 26459 raft_consensus.cc:2229] T b12cd0a6ba3c4d73ae671b6b81f71c3f P a09691f64db94f8d80a439640268f442 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 15:15:32.991811 26459 raft_consensus.cc:2258] T b12cd0a6ba3c4d73ae671b6b81f71c3f P a09691f64db94f8d80a439640268f442 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:15:32.995067 26461 tablet_replica.cc:330] T b12cd0a6ba3c4d73ae671b6b81f71c3f P 0c61eae055234692acb4b7dc0250a012: stopping tablet replica
I20260807 15:15:32.995261 26461 raft_consensus.cc:2229] T b12cd0a6ba3c4d73ae671b6b81f71c3f P 0c61eae055234692acb4b7dc0250a012 [term 1 LEADER]: Raft consensus shutting down.
I20260807 15:15:32.995428 26461 raft_consensus.cc:2258] T b12cd0a6ba3c4d73ae671b6b81f71c3f P 0c61eae055234692acb4b7dc0250a012 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:15:33.000053 26461 ts_tablet_manager.cc:1905] T b12cd0a6ba3c4d73ae671b6b81f71c3f P 0c61eae055234692acb4b7dc0250a012: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:15:33.001413 26460 tablet_replica.cc:330] T b12cd0a6ba3c4d73ae671b6b81f71c3f P 667d880eb4cb4bec8737914e2b398f96: stopping tablet replica
I20260807 15:15:33.001605 26460 raft_consensus.cc:2229] T b12cd0a6ba3c4d73ae671b6b81f71c3f P 667d880eb4cb4bec8737914e2b398f96 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 15:15:33.001724 26460 raft_consensus.cc:2258] T b12cd0a6ba3c4d73ae671b6b81f71c3f P 667d880eb4cb4bec8737914e2b398f96 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:15:33.002257 26460 ts_tablet_manager.cc:1905] T b12cd0a6ba3c4d73ae671b6b81f71c3f P 667d880eb4cb4bec8737914e2b398f96: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:15:33.004051 26460 ts_tablet_manager.cc:1918] T b12cd0a6ba3c4d73ae671b6b81f71c3f P 667d880eb4cb4bec8737914e2b398f96: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 15:15:33.004150 26460 log.cc:1196] T b12cd0a6ba3c4d73ae671b6b81f71c3f P 667d880eb4cb4bec8737914e2b398f96: Deleting WAL directory at /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115480851507-30472-0/minicluster-data/ts-1/wal/wals/b12cd0a6ba3c4d73ae671b6b81f71c3f
I20260807 15:15:33.004467 26460 ts_tablet_manager.cc:1939] T b12cd0a6ba3c4d73ae671b6b81f71c3f P 667d880eb4cb4bec8737914e2b398f96: Deleting consensus metadata
I20260807 15:15:33.005028 25714 catalog_manager.cc:4841] TS 667d880eb4cb4bec8737914e2b398f96 (127.29.194.2:46075): tablet b12cd0a6ba3c4d73ae671b6b81f71c3f (table test-table [id=ed2b49a247bb426082584fff22677234]) successfully deleted
I20260807 15:15:33.005193 26461 ts_tablet_manager.cc:1918] T b12cd0a6ba3c4d73ae671b6b81f71c3f P 0c61eae055234692acb4b7dc0250a012: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 15:15:33.005275 26461 log.cc:1196] T b12cd0a6ba3c4d73ae671b6b81f71c3f P 0c61eae055234692acb4b7dc0250a012: Deleting WAL directory at /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115480851507-30472-0/minicluster-data/ts-2/wal/wals/b12cd0a6ba3c4d73ae671b6b81f71c3f
I20260807 15:15:33.005544 26461 ts_tablet_manager.cc:1939] T b12cd0a6ba3c4d73ae671b6b81f71c3f P 0c61eae055234692acb4b7dc0250a012: Deleting consensus metadata
I20260807 15:15:33.005963 25715 catalog_manager.cc:4841] TS 0c61eae055234692acb4b7dc0250a012 (127.29.194.3:35727): tablet b12cd0a6ba3c4d73ae671b6b81f71c3f (table test-table [id=ed2b49a247bb426082584fff22677234]) successfully deleted
W20260807 15:15:33.013763 25712 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.29.194.1:33343 (error 108)
I20260807 15:15:33.013923 30472 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7fZSIL/build/release/bin/kudu with pid 26017
I20260807 15:15:33.018759 26415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:33.017787 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.1:33343 (local address 127.0.0.1:58438)
0807 15:15:33.018641 (+ 854us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:33.018710 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.194.1:33343: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":825}
W20260807 15:15:33.018957 25712 catalog_manager.cc:4584] TS a09691f64db94f8d80a439640268f442 (127.29.194.1:33343): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet b12cd0a6ba3c4d73ae671b6b81f71c3f: Network error: Client connection negotiation failed: client connection to 127.29.194.1:33343: connect: Connection refused (error 111)
I20260807 15:15:33.039424 30472 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7fZSIL/build/release/bin/kudu with pid 26211
I20260807 15:15:33.056193 30472 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7fZSIL/build/release/bin/kudu with pid 25683
2026-08-07T15:15:33Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (810 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115480851507-30472-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 15:15:33 dist-test-slave-42z9 krb5kdc[26466](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 15:15:33 dist-test-slave-42z9 krb5kdc[26466](info): set up 2 sockets
Aug 07 15:15:33 dist-test-slave-42z9 krb5kdc[26466](info): commencing operation
krb5kdc: starting...
W20260807 15:15:34.196606 30472 mini_kdc.cc:121] Time spent starting KDC: real 1.114s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115480851507-30472-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115480851507-30472-0/krb5kdc/test-user.keytab.
Aug 07 15:15:34 dist-test-slave-42z9 krb5kdc[26466](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115734, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T15:15:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T15:15:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.194.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.194.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.194.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.62.keytab.
Entry for principal kudu/127.29.194.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.62.keytab.
I20260807 15:15:34.322101 30472 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7fZSIL/build/release/bin/kudu
/tmp/dist-test-task7fZSIL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115480851507-30472-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115480851507-30472-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115480851507-30472-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115480851507-30472-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.194.62:42211
--webserver_interface=127.29.194.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.62.keytab
--principal=kudu/127.29.194.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.194.20:45989
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.194.62:42211
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115480851507-30472-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115480851507-30472-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115480851507-30472-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:15:34.405117 26486 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:15:34.405352 26486 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:15:34.405383 26486 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:15:34.406612 26486 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:15:34.406661 26486 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:15:34.406679 26486 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:15:34.406697 26486 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:15:34.406713 26486 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:15:34.408346 26486 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.194.20:45989
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115480851507-30472-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115480851507-30472-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.194.62:42211
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.62.keytab
--principal=kudu/127.29.194.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.194.62:42211
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115480851507-30472-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.194.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26486
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115480851507-30472-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:15:34.408614 26486 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:15:34.408830 26486 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:15:34.413764 26499 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:15:34.413769 26494 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:15:34.413995 26486 server_base.cc:1034] running on GCE node
W20260807 15:15:34.413765 26495 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:15:34.414278 26486 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:15:34.414558 26486 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:15:34.415719 26486 hybrid_clock.cc:648] HybridClock initialized: now 1786115734415703 us; error 31 us; skew 500 ppm
Aug 07 15:15:34 dist-test-slave-42z9 krb5kdc[26466](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115734, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.194.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:15:34.418300 26486 init.cc:377] Logged in from keytab as kudu/127.29.194.62@KRBTEST.COM (short username kudu)
I20260807 15:15:34.418783 26486 webserver.cc:460] Webserver started at http://127.29.194.62:44789/ using document root <none> and password file <none>
I20260807 15:15:34.419003 26486 fs_manager.cc:359] Metadata directory not provided
I20260807 15:15:34.419050 26486 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:15:34.419142 26486 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:15:34.420372 26486 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115480851507-30472-0/minicluster-data/master-0/data/instance:
uuid: "ffdbfd49a4734e86af35d0ab8ae866fc"
format_stamp: "Formatted at 2026-08-07 15:15:34 on dist-test-slave-42z9"
server_key: "95e6d1b4bb0252215285627390178c6b"
server_key_iv: "fda8cd302ad0c3f1bbf5deb4bd26d530"
server_key_version: "encryptionkey@0"
I20260807 15:15:34.420691 26486 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115480851507-30472-0/minicluster-data/master-0/wal/instance:
uuid: "ffdbfd49a4734e86af35d0ab8ae866fc"
format_stamp: "Formatted at 2026-08-07 15:15:34 on dist-test-slave-42z9"
server_key: "95e6d1b4bb0252215285627390178c6b"
server_key_iv: "fda8cd302ad0c3f1bbf5deb4bd26d530"
server_key_version: "encryptionkey@0"
I20260807 15:15:34.422065 26486 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 15:15:34.423138 26507 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:15:34.423498 26486 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:15:34.423578 26486 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115480851507-30472-0/minicluster-data/master-0/data,/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115480851507-30472-0/minicluster-data/master-0/wal
uuid: "ffdbfd49a4734e86af35d0ab8ae866fc"
format_stamp: "Formatted at 2026-08-07 15:15:34 on dist-test-slave-42z9"
server_key: "95e6d1b4bb0252215285627390178c6b"
server_key_iv: "fda8cd302ad0c3f1bbf5deb4bd26d530"
server_key_version: "encryptionkey@0"
I20260807 15:15:34.423897 26486 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115480851507-30472-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115480851507-30472-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115480851507-30472-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:15:34.446491 26486 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:15:34.449478 26486 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:15:34.449815 26486 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:15:34.454100 26486 rpc_server.cc:307] RPC server started. Bound to: 127.29.194.62:42211
I20260807 15:15:34.454144 26588 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.194.62:42211 every 8 connection(s)
I20260807 15:15:34.454509 26486 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115480851507-30472-0/minicluster-data/master-0/data/info.pb
I20260807 15:15:34.455466 26590 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:15:34.458237 30472 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7fZSIL/build/release/bin/kudu as pid 26486
I20260807 15:15:34.458377 30472 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115480851507-30472-0/minicluster-data/master-0/wal/instance
I20260807 15:15:34.458590 30472 external_mini_cluster.cc:1428] Setting key bfccfb9e9128780b78af4859ba3da641
I20260807 15:15:34.458773 26590 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ffdbfd49a4734e86af35d0ab8ae866fc: Bootstrap starting.
I20260807 15:15:34.459713 26590 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ffdbfd49a4734e86af35d0ab8ae866fc: Neither blocks nor log segments found. Creating new log.
I20260807 15:15:34.460093 26590 log.cc:824] T 00000000000000000000000000000000 P ffdbfd49a4734e86af35d0ab8ae866fc: Log is configured to *not* fsync() on all Append() calls
I20260807 15:15:34.460958 26590 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ffdbfd49a4734e86af35d0ab8ae866fc: No bootstrap required, opened a new log
I20260807 15:15:34.463459 26590 raft_consensus.cc:348] T 00000000000000000000000000000000 P ffdbfd49a4734e86af35d0ab8ae866fc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ffdbfd49a4734e86af35d0ab8ae866fc" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 42211 } }
I20260807 15:15:34.463621 26590 raft_consensus.cc:374] T 00000000000000000000000000000000 P ffdbfd49a4734e86af35d0ab8ae866fc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:15:34.463644 26590 raft_consensus.cc:729] T 00000000000000000000000000000000 P ffdbfd49a4734e86af35d0ab8ae866fc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ffdbfd49a4734e86af35d0ab8ae866fc, State: Initialized, Role: FOLLOWER
I20260807 15:15:34.463760 26590 consensus_queue.cc:260] T 00000000000000000000000000000000 P ffdbfd49a4734e86af35d0ab8ae866fc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ffdbfd49a4734e86af35d0ab8ae866fc" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 42211 } }
I20260807 15:15:34.463835 26590 raft_consensus.cc:388] T 00000000000000000000000000000000 P ffdbfd49a4734e86af35d0ab8ae866fc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:15:34.463868 26590 raft_consensus.cc:482] T 00000000000000000000000000000000 P ffdbfd49a4734e86af35d0ab8ae866fc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:15:34.463912 26590 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ffdbfd49a4734e86af35d0ab8ae866fc [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:15:34.464877 26590 raft_consensus.cc:504] T 00000000000000000000000000000000 P ffdbfd49a4734e86af35d0ab8ae866fc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ffdbfd49a4734e86af35d0ab8ae866fc" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 42211 } }
I20260807 15:15:34.465004 26590 leader_election.cc:302] T 00000000000000000000000000000000 P ffdbfd49a4734e86af35d0ab8ae866fc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ffdbfd49a4734e86af35d0ab8ae866fc; no voters:
I20260807 15:15:34.465181 26590 leader_election.cc:288] T 00000000000000000000000000000000 P ffdbfd49a4734e86af35d0ab8ae866fc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 15:15:34.465313 26599 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ffdbfd49a4734e86af35d0ab8ae866fc [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:15:34.465530 26590 sys_catalog.cc:564] T 00000000000000000000000000000000 P ffdbfd49a4734e86af35d0ab8ae866fc [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:15:34.465586 26599 raft_consensus.cc:686] T 00000000000000000000000000000000 P ffdbfd49a4734e86af35d0ab8ae866fc [term 1 LEADER]: Becoming Leader. State: Replica: ffdbfd49a4734e86af35d0ab8ae866fc, State: Running, Role: LEADER
I20260807 15:15:34.465780 26599 consensus_queue.cc:237] T 00000000000000000000000000000000 P ffdbfd49a4734e86af35d0ab8ae866fc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ffdbfd49a4734e86af35d0ab8ae866fc" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 42211 } }
I20260807 15:15:34.466387 26599 sys_catalog.cc:455] T 00000000000000000000000000000000 P ffdbfd49a4734e86af35d0ab8ae866fc [sys.catalog]: SysCatalogTable state changed. Reason: New leader ffdbfd49a4734e86af35d0ab8ae866fc. Latest consensus state: current_term: 1 leader_uuid: "ffdbfd49a4734e86af35d0ab8ae866fc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ffdbfd49a4734e86af35d0ab8ae866fc" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 42211 } } }
I20260807 15:15:34.466507 26599 sys_catalog.cc:458] T 00000000000000000000000000000000 P ffdbfd49a4734e86af35d0ab8ae866fc [sys.catalog]: This master's current role is: LEADER
I20260807 15:15:34.466764 26615 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:15:34.467335 26601 sys_catalog.cc:455] T 00000000000000000000000000000000 P ffdbfd49a4734e86af35d0ab8ae866fc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ffdbfd49a4734e86af35d0ab8ae866fc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ffdbfd49a4734e86af35d0ab8ae866fc" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 42211 } } }
Aug 07 15:15:34 dist-test-slave-42z9 krb5kdc[26466](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115734, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.194.62@KRBTEST.COM
I20260807 15:15:34.467393 26615 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:15:34.467765 26601 sys_catalog.cc:458] T 00000000000000000000000000000000 P ffdbfd49a4734e86af35d0ab8ae866fc [sys.catalog]: This master's current role is: LEADER
I20260807 15:15:34.470779 26615 catalog_manager.cc:1347] Generated new cluster ID: 837a86f45e4e4f0e9d31c3e34be794b0
I20260807 15:15:34.470842 26615 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:15:34.472259 26597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:34.459423 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41578 (local address 127.29.194.62:42211)
0807 15:15:34.459737 (+ 314us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:34.459747 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:34.459792 (+ 45us) server_negotiation.cc:407] Connection header received
0807 15:15:34.460516 (+ 724us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:34.460534 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:34.460596 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:34.460771 (+ 175us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:15:34.461499 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:34.462929 (+ 1430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:34.464118 (+ 1189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:34.464725 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:34.464851 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:34.468364 (+ 3513us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:15:34.468369 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:15:34.468369 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:15:34.468384 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:15:34.470475 (+ 2091us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:15:34.470899 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:15:34.470903 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:15:34.470903 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:15:34.471039 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:15:34.471288 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:15:34.471288 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:15:34.471289 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:15:34.471600 (+ 311us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:15:34.471762 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0807 15:15:34.471884 (+ 122us) server_negotiation.cc:299] Negotiation successful
0807 15:15:34.472041 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":239,"thread_start_us":145,"threads_started":1}
I20260807 15:15:34.478955 26615 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:15:34.479548 26615 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:15:34.483369 26615 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ffdbfd49a4734e86af35d0ab8ae866fc: Generated new TSK 0
I20260807 15:15:34.483590 26615 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.29.194.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.194.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.194.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.1.keytab.
Entry for principal kudu/127.29.194.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.1.keytab.
I20260807 15:15:34.521518 30472 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7fZSIL/build/release/bin/kudu
/tmp/dist-test-task7fZSIL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115480851507-30472-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115480851507-30472-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115480851507-30472-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115480851507-30472-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.194.1:0
--local_ip_for_outbound_sockets=127.29.194.1
--webserver_interface=127.29.194.1
--webserver_port=0
--tserver_master_addrs=127.29.194.62:42211
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.1.keytab
--principal=kudu/127.29.194.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.194.20:45989
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115480851507-30472-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115480851507-30472-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115480851507-30472-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:15:34.621522 26632 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:15:34.621762 26632 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:15:34.621789 26632 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:15:34.623035 26632 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:15:34.623090 26632 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:15:34.623117 26632 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.194.1
I20260807 15:15:34.624709 26632 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.194.20:45989
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115480851507-30472-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115480851507-30472-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.1.keytab
--principal=kudu/127.29.194.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.194.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115480851507-30472-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.194.1
--webserver_port=0
--tserver_master_addrs=127.29.194.62:42211
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26632
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.194.1
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115480851507-30472-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:15:34.624951 26632 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:15:34.625145 26632 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:15:34.630812 26641 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:15:34.630816 26644 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:15:34.630877 26640 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:15:34.630961 26632 server_base.cc:1034] running on GCE node
I20260807 15:15:34.631280 26632 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:15:34.631520 26632 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:15:34.632679 26632 hybrid_clock.cc:648] HybridClock initialized: now 1786115734632660 us; error 38 us; skew 500 ppm
Aug 07 15:15:34 dist-test-slave-42z9 krb5kdc[26466](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115734, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.194.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:15:34.635800 26632 init.cc:377] Logged in from keytab as kudu/127.29.194.1@KRBTEST.COM (short username kudu)
I20260807 15:15:34.636423 26632 webserver.cc:460] Webserver started at http://127.29.194.1:46735/ using document root <none> and password file <none>
I20260807 15:15:34.636694 26632 fs_manager.cc:359] Metadata directory not provided
I20260807 15:15:34.636749 26632 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:15:34.636909 26632 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:15:34.638381 26632 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115480851507-30472-0/minicluster-data/ts-0/data/instance:
uuid: "89d925f6e24b4f768e619cde5f376652"
format_stamp: "Formatted at 2026-08-07 15:15:34 on dist-test-slave-42z9"
server_key: "18e0576dd8fd0fa22c347047cff95dbc"
server_key_iv: "a183308e0979af8b8f92480c6302eb68"
server_key_version: "encryptionkey@0"
I20260807 15:15:34.638787 26632 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115480851507-30472-0/minicluster-data/ts-0/wal/instance:
uuid: "89d925f6e24b4f768e619cde5f376652"
format_stamp: "Formatted at 2026-08-07 15:15:34 on dist-test-slave-42z9"
server_key: "18e0576dd8fd0fa22c347047cff95dbc"
server_key_iv: "a183308e0979af8b8f92480c6302eb68"
server_key_version: "encryptionkey@0"
I20260807 15:15:34.640324 26632 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260807 15:15:34.641270 26653 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:15:34.641563 26632 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 15:15:34.641666 26632 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115480851507-30472-0/minicluster-data/ts-0/data,/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115480851507-30472-0/minicluster-data/ts-0/wal
uuid: "89d925f6e24b4f768e619cde5f376652"
format_stamp: "Formatted at 2026-08-07 15:15:34 on dist-test-slave-42z9"
server_key: "18e0576dd8fd0fa22c347047cff95dbc"
server_key_iv: "a183308e0979af8b8f92480c6302eb68"
server_key_version: "encryptionkey@0"
I20260807 15:15:34.641745 26632 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115480851507-30472-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115480851507-30472-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115480851507-30472-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:15:34.653250 26632 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:15:34.655580 26632 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:15:34.655710 26632 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:15:34.655954 26632 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:15:34.656291 26632 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:15:34.656325 26632 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:15:34.656355 26632 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:15:34.656380 26632 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:15:34.662259 26632 rpc_server.cc:307] RPC server started. Bound to: 127.29.194.1:36245
I20260807 15:15:34.662329 26822 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.194.1:36245 every 8 connection(s)
I20260807 15:15:34.662639 26632 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115480851507-30472-0/minicluster-data/ts-0/data/info.pb
I20260807 15:15:34.668915 30472 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7fZSIL/build/release/bin/kudu as pid 26632
I20260807 15:15:34.669025 30472 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115480851507-30472-0/minicluster-data/ts-0/wal/instance
I20260807 15:15:34.669235 30472 external_mini_cluster.cc:1428] Setting key 32ca7d47f2d72588061e5a6de5d37796
Aug 07 15:15:34 dist-test-slave-42z9 krb5kdc[26466](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115734, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.194.1@KRBTEST.COM for kudu/127.29.194.62@KRBTEST.COM
I20260807 15:15:34.673831 26597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:34.663807 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.194.1:36965 (local address 127.29.194.62:42211)
0807 15:15:34.663889 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:34.663892 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:34.664070 (+ 178us) server_negotiation.cc:407] Connection header received
0807 15:15:34.664776 (+ 706us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:34.664778 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:34.664795 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:34.664863 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:15:34.665984 (+ 1121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:34.666574 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:34.667644 (+ 1070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:34.667878 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:34.667937 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:34.670474 (+ 2537us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:15:34.670476 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:15:34.670476 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:15:34.670484 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:15:34.671991 (+ 1507us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:15:34.672526 (+ 535us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:15:34.672529 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:15:34.672529 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:15:34.672732 (+ 203us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:15:34.673041 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:15:34.673042 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:15:34.673042 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:15:34.673197 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:15:34.673253 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0807 15:15:34.673619 (+ 366us) server_negotiation.cc:299] Negotiation successful
0807 15:15:34.673733 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260807 15:15:34.673995 26826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:34.663621 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:42211 (local address 127.29.194.1:36965)
0807 15:15:34.663878 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:34.663955 (+ 77us) client_negotiation.cc:174] Beginning negotiation
0807 15:15:34.664604 (+ 649us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:15:34.664942 (+ 338us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:15:34.664947 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:15:34.665660 (+ 713us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:15:34.665878 (+ 218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:34.665883 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:34.666668 (+ 785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:34.666669 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:34.667561 (+ 892us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:34.667565 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:34.667956 (+ 391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:34.667957 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:34.668078 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:34.668601 (+ 523us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:15:34.668610 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:15:34.670355 (+ 1745us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:15:34.672101 (+ 1746us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:15:34.672110 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:15:34.672112 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:15:34.672371 (+ 259us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:15:34.672832 (+ 461us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:15:34.672833 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:15:34.672834 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:15:34.672967 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:15:34.673266 (+ 299us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:15:34.673280 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:15:34.673422 (+ 142us) client_negotiation.cc:769] Sending connection context
0807 15:15:34.673576 (+ 154us) client_negotiation.cc:240] Negotiation successful
0807 15:15:34.673766 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":100,"threads_started":1}
I20260807 15:15:34.674497 26824 heartbeater.cc:347] Connected to a master server at 127.29.194.62:42211
I20260807 15:15:34.674594 26824 heartbeater.cc:464] Registering TS with master...
I20260807 15:15:34.674772 26824 heartbeater.cc:511] Master 127.29.194.62:42211 requested a full tablet report, sending...
I20260807 15:15:34.675275 26532 ts_manager.cc:194] Registered new tserver with Master: 89d925f6e24b4f768e619cde5f376652 (127.29.194.1:36245)
I20260807 15:15:34.675895 26532 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.194.1@KRBTEST.COM'} at 127.29.194.1:36965
WARNING: no policy specified for kudu/127.29.194.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.194.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.194.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.2.keytab.
Entry for principal kudu/127.29.194.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.2.keytab.
I20260807 15:15:34.701067 30472 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7fZSIL/build/release/bin/kudu
/tmp/dist-test-task7fZSIL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115480851507-30472-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115480851507-30472-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115480851507-30472-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115480851507-30472-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.194.2:0
--local_ip_for_outbound_sockets=127.29.194.2
--webserver_interface=127.29.194.2
--webserver_port=0
--tserver_master_addrs=127.29.194.62:42211
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.2.keytab
--principal=kudu/127.29.194.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.194.20:45989
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115480851507-30472-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115480851507-30472-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115480851507-30472-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:15:34.784898 26829 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:15:34.785118 26829 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:15:34.785153 26829 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:15:34.786345 26829 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:15:34.786392 26829 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:15:34.786428 26829 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.194.2
I20260807 15:15:34.789793 26829 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.194.20:45989
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115480851507-30472-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115480851507-30472-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.2.keytab
--principal=kudu/127.29.194.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.194.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115480851507-30472-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.194.2
--webserver_port=0
--tserver_master_addrs=127.29.194.62:42211
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26829
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.194.2
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115480851507-30472-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:15:34.790076 26829 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:15:34.790311 26829 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:15:34.795329 26836 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:15:34.795476 26842 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:15:34.795580 26838 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:15:34.795578 26829 server_base.cc:1034] running on GCE node
I20260807 15:15:34.797399 26829 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:15:34.797695 26829 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:15:34.798868 26829 hybrid_clock.cc:648] HybridClock initialized: now 1786115734798855 us; error 29 us; skew 500 ppm
Aug 07 15:15:34 dist-test-slave-42z9 krb5kdc[26466](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115734, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.194.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:15:34.801378 26829 init.cc:377] Logged in from keytab as kudu/127.29.194.2@KRBTEST.COM (short username kudu)
I20260807 15:15:34.801838 26829 webserver.cc:460] Webserver started at http://127.29.194.2:42153/ using document root <none> and password file <none>
I20260807 15:15:34.802064 26829 fs_manager.cc:359] Metadata directory not provided
I20260807 15:15:34.802109 26829 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:15:34.802237 26829 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:15:34.803293 26829 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115480851507-30472-0/minicluster-data/ts-1/data/instance:
uuid: "6f999502eed741cebab3c92e231b4483"
format_stamp: "Formatted at 2026-08-07 15:15:34 on dist-test-slave-42z9"
server_key: "6b38c5c397757bfc9cb5084880df84a1"
server_key_iv: "58308a2ebc30b0c9f7ffde327af74e26"
server_key_version: "encryptionkey@0"
I20260807 15:15:34.803625 26829 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115480851507-30472-0/minicluster-data/ts-1/wal/instance:
uuid: "6f999502eed741cebab3c92e231b4483"
format_stamp: "Formatted at 2026-08-07 15:15:34 on dist-test-slave-42z9"
server_key: "6b38c5c397757bfc9cb5084880df84a1"
server_key_iv: "58308a2ebc30b0c9f7ffde327af74e26"
server_key_version: "encryptionkey@0"
I20260807 15:15:34.805078 26829 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 15:15:34.806008 26853 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:15:34.806257 26829 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260807 15:15:34.806334 26829 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115480851507-30472-0/minicluster-data/ts-1/data,/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115480851507-30472-0/minicluster-data/ts-1/wal
uuid: "6f999502eed741cebab3c92e231b4483"
format_stamp: "Formatted at 2026-08-07 15:15:34 on dist-test-slave-42z9"
server_key: "6b38c5c397757bfc9cb5084880df84a1"
server_key_iv: "58308a2ebc30b0c9f7ffde327af74e26"
server_key_version: "encryptionkey@0"
I20260807 15:15:34.806422 26829 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115480851507-30472-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115480851507-30472-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115480851507-30472-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:15:34.838478 26829 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:15:34.840525 26829 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:15:34.840649 26829 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:15:34.840898 26829 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:15:34.841203 26829 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:15:34.841236 26829 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:15:34.841259 26829 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:15:34.841281 26829 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:15:34.847173 26829 rpc_server.cc:307] RPC server started. Bound to: 127.29.194.2:46529
I20260807 15:15:34.847229 27027 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.194.2:46529 every 8 connection(s)
I20260807 15:15:34.847581 26829 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115480851507-30472-0/minicluster-data/ts-1/data/info.pb
I20260807 15:15:34.848263 30472 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7fZSIL/build/release/bin/kudu as pid 26829
I20260807 15:15:34.848387 30472 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115480851507-30472-0/minicluster-data/ts-1/wal/instance
I20260807 15:15:34.848604 30472 external_mini_cluster.cc:1428] Setting key 4112efe9bd5f51d6b69f2262aaf5ae8b
Aug 07 15:15:34 dist-test-slave-42z9 krb5kdc[26466](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115734, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.194.2@KRBTEST.COM for kudu/127.29.194.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.29.194.3@KRBTEST.COM; defaulting to no policy
I20260807 15:15:34.857652 26597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:34.848788 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.194.2:53941 (local address 127.29.194.62:42211)
0807 15:15:34.848847 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:34.848850 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:34.849199 (+ 349us) server_negotiation.cc:407] Connection header received
0807 15:15:34.849811 (+ 612us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:34.849813 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:34.849828 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:34.849904 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:15:34.850928 (+ 1024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:34.851640 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:34.852669 (+ 1029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:34.852868 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:34.852925 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:34.855074 (+ 2149us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:15:34.855075 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:15:34.855076 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:15:34.855083 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:15:34.856338 (+ 1255us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:15:34.856717 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:15:34.856719 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:15:34.856719 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:15:34.856775 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:15:34.856993 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:15:34.856994 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:15:34.856994 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:15:34.857118 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:15:34.857174 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0807 15:15:34.857470 (+ 296us) server_negotiation.cc:299] Negotiation successful
0807 15:15:34.857558 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260807 15:15:34.857832 27033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:34.848768 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:42211 (local address 127.29.194.2:53941)
0807 15:15:34.849035 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:34.849079 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0807 15:15:34.849691 (+ 612us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:15:34.849974 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:15:34.849977 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:15:34.850581 (+ 604us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:15:34.850789 (+ 208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:34.850794 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:34.851757 (+ 963us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:34.851758 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:34.852577 (+ 819us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:34.852580 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:34.852941 (+ 361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:34.852942 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:34.853070 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:34.853626 (+ 556us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:15:34.853635 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:15:34.854978 (+ 1343us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:15:34.856414 (+ 1436us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:15:34.856421 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:15:34.856424 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:15:34.856650 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:15:34.856828 (+ 178us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:15:34.856829 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:15:34.856829 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:15:34.856935 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:15:34.857176 (+ 241us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:15:34.857183 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:15:34.857322 (+ 139us) client_negotiation.cc:769] Sending connection context
0807 15:15:34.857414 (+ 92us) client_negotiation.cc:240] Negotiation successful
0807 15:15:34.857616 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":213,"thread_start_us":144,"threads_started":1}
I20260807 15:15:34.858297 27028 heartbeater.cc:347] Connected to a master server at 127.29.194.62:42211
I20260807 15:15:34.858399 27028 heartbeater.cc:464] Registering TS with master...
I20260807 15:15:34.858583 27028 heartbeater.cc:511] Master 127.29.194.62:42211 requested a full tablet report, sending...
I20260807 15:15:34.858963 26532 ts_manager.cc:194] Registered new tserver with Master: 6f999502eed741cebab3c92e231b4483 (127.29.194.2:46529)
I20260807 15:15:34.859328 26532 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.194.2@KRBTEST.COM'} at 127.29.194.2:53941
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.194.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.194.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.3.keytab.
Entry for principal kudu/127.29.194.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.3.keytab.
I20260807 15:15:34.875684 30472 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7fZSIL/build/release/bin/kudu
/tmp/dist-test-task7fZSIL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115480851507-30472-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115480851507-30472-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115480851507-30472-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115480851507-30472-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.194.3:0
--local_ip_for_outbound_sockets=127.29.194.3
--webserver_interface=127.29.194.3
--webserver_port=0
--tserver_master_addrs=127.29.194.62:42211
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.3.keytab
--principal=kudu/127.29.194.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.194.20:45989
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115480851507-30472-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115480851507-30472-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115480851507-30472-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:15:34.961773 27037 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:15:34.962010 27037 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:15:34.962038 27037 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:15:34.963297 27037 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:15:34.963346 27037 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:15:34.963374 27037 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.194.3
I20260807 15:15:34.964895 27037 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.194.20:45989
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115480851507-30472-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115480851507-30472-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.3.keytab
--principal=kudu/127.29.194.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.194.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115480851507-30472-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.194.3
--webserver_port=0
--tserver_master_addrs=127.29.194.62:42211
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27037
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.194.3
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115480851507-30472-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:15:34.965116 27037 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:15:34.965371 27037 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:15:34.970712 27042 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:15:34.970769 27037 server_base.cc:1034] running on GCE node
W20260807 15:15:34.970737 27043 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:15:34.970749 27045 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:15:34.971134 27037 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:15:34.971395 27037 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:15:34.972546 27037 hybrid_clock.cc:648] HybridClock initialized: now 1786115734972541 us; error 32 us; skew 500 ppm
Aug 07 15:15:34 dist-test-slave-42z9 krb5kdc[26466](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115734, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.194.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:15:34.975095 27037 init.cc:377] Logged in from keytab as kudu/127.29.194.3@KRBTEST.COM (short username kudu)
I20260807 15:15:34.975534 27037 webserver.cc:460] Webserver started at http://127.29.194.3:33297/ using document root <none> and password file <none>
I20260807 15:15:34.975718 27037 fs_manager.cc:359] Metadata directory not provided
I20260807 15:15:34.975757 27037 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:15:34.975853 27037 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:15:34.976890 27037 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115480851507-30472-0/minicluster-data/ts-2/data/instance:
uuid: "bede7706385c4e17b2b9bafb10a0398f"
format_stamp: "Formatted at 2026-08-07 15:15:34 on dist-test-slave-42z9"
server_key: "a526b3ec03596a0107e8fd4cfb2cf1e4"
server_key_iv: "bbb67616ec9f58e15d32ed619e6b8172"
server_key_version: "encryptionkey@0"
I20260807 15:15:34.977201 27037 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115480851507-30472-0/minicluster-data/ts-2/wal/instance:
uuid: "bede7706385c4e17b2b9bafb10a0398f"
format_stamp: "Formatted at 2026-08-07 15:15:34 on dist-test-slave-42z9"
server_key: "a526b3ec03596a0107e8fd4cfb2cf1e4"
server_key_iv: "bbb67616ec9f58e15d32ed619e6b8172"
server_key_version: "encryptionkey@0"
I20260807 15:15:34.978585 27037 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 15:15:34.979460 27056 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:15:34.979849 27037 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:15:34.979990 27037 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115480851507-30472-0/minicluster-data/ts-2/data,/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115480851507-30472-0/minicluster-data/ts-2/wal
uuid: "bede7706385c4e17b2b9bafb10a0398f"
format_stamp: "Formatted at 2026-08-07 15:15:34 on dist-test-slave-42z9"
server_key: "a526b3ec03596a0107e8fd4cfb2cf1e4"
server_key_iv: "bbb67616ec9f58e15d32ed619e6b8172"
server_key_version: "encryptionkey@0"
I20260807 15:15:34.980091 27037 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115480851507-30472-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115480851507-30472-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115480851507-30472-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:15:34.991562 27037 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:15:34.993685 27037 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:15:34.993801 27037 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:15:34.994005 27037 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:15:34.994272 27037 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:15:34.994302 27037 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:15:34.994323 27037 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:15:34.994334 27037 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:15:35.000430 27037 rpc_server.cc:307] RPC server started. Bound to: 127.29.194.3:38613
I20260807 15:15:35.000490 27225 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.194.3:38613 every 8 connection(s)
I20260807 15:15:35.000805 27037 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115480851507-30472-0/minicluster-data/ts-2/data/info.pb
I20260807 15:15:35.001781 30472 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7fZSIL/build/release/bin/kudu as pid 27037
I20260807 15:15:35.001914 30472 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115480851507-30472-0/minicluster-data/ts-2/wal/instance
I20260807 15:15:35.002175 30472 external_mini_cluster.cc:1428] Setting key 8f0c99c62973402b2dc2d766d106dbce
Aug 07 15:15:35 dist-test-slave-42z9 krb5kdc[26466](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115734, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.194.3@KRBTEST.COM for kudu/127.29.194.62@KRBTEST.COM
I20260807 15:15:35.011448 26597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:35.001637 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.194.3:33081 (local address 127.29.194.62:42211)
0807 15:15:35.001720 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:35.001723 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:35.001934 (+ 211us) server_negotiation.cc:407] Connection header received
0807 15:15:35.002572 (+ 638us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:35.002574 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:35.002587 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:35.002672 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:15:35.003800 (+ 1128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:35.004314 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:35.005394 (+ 1080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:35.005742 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:35.005851 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:35.008782 (+ 2931us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:15:35.008784 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:15:35.008785 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:15:35.008794 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:15:35.010164 (+ 1370us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:15:35.010633 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:15:35.010634 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:15:35.010635 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:15:35.010688 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:15:35.010912 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:15:35.010913 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:15:35.010913 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:15:35.011031 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:15:35.011081 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0807 15:15:35.011306 (+ 225us) server_negotiation.cc:299] Negotiation successful
0807 15:15:35.011378 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260807 15:15:35.011651 27229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:35.001542 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:42211 (local address 127.29.194.3:33081)
0807 15:15:35.001786 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:35.001826 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0807 15:15:35.002427 (+ 601us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:15:35.002762 (+ 335us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:15:35.002767 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:15:35.003486 (+ 719us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:15:35.003694 (+ 208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:35.003699 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:35.004425 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:35.004426 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:35.005253 (+ 827us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:35.005256 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:35.005880 (+ 624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:35.005881 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:35.005996 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:35.006657 (+ 661us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:15:35.006670 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:15:35.008672 (+ 2002us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:15:35.010249 (+ 1577us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:15:35.010256 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:15:35.010258 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:15:35.010573 (+ 315us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:15:35.010746 (+ 173us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:15:35.010746 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:15:35.010747 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:15:35.010862 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:15:35.011078 (+ 216us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:15:35.011082 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:15:35.011208 (+ 126us) client_negotiation.cc:769] Sending connection context
0807 15:15:35.011289 (+ 81us) client_negotiation.cc:240] Negotiation successful
0807 15:15:35.011425 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":82,"threads_started":1}
I20260807 15:15:35.012180 27226 heartbeater.cc:347] Connected to a master server at 127.29.194.62:42211
I20260807 15:15:35.012280 27226 heartbeater.cc:464] Registering TS with master...
I20260807 15:15:35.012442 27226 heartbeater.cc:511] Master 127.29.194.62:42211 requested a full tablet report, sending...
I20260807 15:15:35.012806 26532 ts_manager.cc:194] Registered new tserver with Master: bede7706385c4e17b2b9bafb10a0398f (127.29.194.3:38613)
I20260807 15:15:35.013180 26532 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.194.3@KRBTEST.COM'} at 127.29.194.3:33081
I20260807 15:15:35.014025 30472 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 15:15:35.021432 26597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:35.015434 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41586 (local address 127.29.194.62:42211)
0807 15:15:35.015508 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:35.015510 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:35.015702 (+ 192us) server_negotiation.cc:407] Connection header received
0807 15:15:35.015802 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:35.015803 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:35.015817 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:35.015865 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:15:35.016464 (+ 599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:35.016954 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:35.017736 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:35.017932 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:35.017991 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:35.019207 (+ 1216us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:15:35.019208 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:15:35.019208 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:15:35.019215 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:15:35.020330 (+ 1115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:15:35.020636 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:15:35.020637 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:15:35.020637 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:15:35.020682 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:15:35.020923 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:15:35.020924 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:15:35.020925 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:15:35.021101 (+ 176us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:15:35.021196 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0807 15:15:35.021277 (+ 81us) server_negotiation.cc:299] Negotiation successful
0807 15:15:35.021355 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260807 15:15:35.023403 26532 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41586:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 15:15:35.025276 26532 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 15:15:35.031764 27242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:35.027939 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.2:46529 (local address 127.0.0.1:50298)
0807 15:15:35.028078 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:35.028089 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 15:15:35.028149 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:15:35.028525 (+ 376us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:15:35.028526 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:15:35.028527 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:15:35.028559 (+ 32us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:35.028560 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:35.029870 (+ 1310us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:35.029871 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:35.030671 (+ 800us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:35.030673 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:35.031445 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:35.031446 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:35.031527 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:35.031542 (+ 15us) client_negotiation.cc:769] Sending connection context
0807 15:15:35.031622 (+ 80us) client_negotiation.cc:240] Negotiation successful
0807 15:15:35.031676 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":82,"threads_started":1}
I20260807 15:15:35.031766 27248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:35.028180 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50298 (local address 127.29.194.2:46529)
0807 15:15:35.028356 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:35.028361 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:35.028372 (+ 11us) server_negotiation.cc:407] Connection header received
0807 15:15:35.028407 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:35.028409 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:35.028423 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:35.028628 (+ 205us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:15:35.028689 (+ 61us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:35.029753 (+ 1064us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:35.030754 (+ 1001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:35.031336 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:35.031439 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:35.031499 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0807 15:15:35.031643 (+ 144us) server_negotiation.cc:299] Negotiation successful
0807 15:15:35.031683 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137,"thread_start_us":92,"threads_started":1}
I20260807 15:15:35.032265 27241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:35.027744 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.3:38613 (local address 127.0.0.1:55766)
0807 15:15:35.027927 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:35.027967 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0807 15:15:35.028030 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:15:35.028645 (+ 615us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:15:35.028646 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:15:35.028648 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:15:35.028705 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:35.028706 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:35.029722 (+ 1016us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:35.029723 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:35.031034 (+ 1311us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:35.031037 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:35.032075 (+ 1038us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:35.032076 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:35.032143 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:35.032148 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 15:15:35.032178 (+ 30us) client_negotiation.cc:240] Negotiation successful
0807 15:15:35.032216 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":158,"mutex_wait_us":43,"thread_start_us":60,"threads_started":1}
I20260807 15:15:35.032548 27245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:35.027852 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55766 (local address 127.29.194.3:38613)
0807 15:15:35.028482 (+ 630us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:35.028486 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:35.028496 (+ 10us) server_negotiation.cc:407] Connection header received
0807 15:15:35.028532 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:35.028534 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:35.028549 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:35.028775 (+ 226us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:15:35.028839 (+ 64us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:35.029627 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:35.031138 (+ 1511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:35.031973 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:35.032365 (+ 392us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:35.032430 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0807 15:15:35.032455 (+ 25us) server_negotiation.cc:299] Negotiation successful
0807 15:15:35.032491 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":594,"thread_start_us":104,"threads_started":1}
I20260807 15:15:35.032586 26933 tablet_service.cc:1467] Processing CreateTablet for tablet d79d80efce1b41ad8af1182477ff301d (DEFAULT_TABLE table=test-table [id=471f81670f2a416f9d4c0311a1c40513]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:15:35.032964 26933 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d79d80efce1b41ad8af1182477ff301d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:15:35.033118 27132 tablet_service.cc:1467] Processing CreateTablet for tablet d79d80efce1b41ad8af1182477ff301d (DEFAULT_TABLE table=test-table [id=471f81670f2a416f9d4c0311a1c40513]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:15:35.033409 27132 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d79d80efce1b41ad8af1182477ff301d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:15:35.033699 27246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:35.027866 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53192 (local address 127.29.194.1:36245)
0807 15:15:35.028201 (+ 335us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:35.028206 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:35.028217 (+ 11us) server_negotiation.cc:407] Connection header received
0807 15:15:35.028258 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:35.028260 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:35.028275 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:35.028355 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:15:35.028566 (+ 211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:35.029734 (+ 1168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:35.031126 (+ 1392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:35.031800 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:35.031867 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:35.031921 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0807 15:15:35.033570 (+ 1649us) server_negotiation.cc:299] Negotiation successful
0807 15:15:35.033623 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":294,"thread_start_us":109,"threads_started":1}
I20260807 15:15:35.033948 27247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:35.027744 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.1:36245 (local address 127.0.0.1:53192)
0807 15:15:35.028078 (+ 334us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:35.028089 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 15:15:35.028149 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:15:35.028394 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:15:35.028397 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:15:35.028400 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:15:35.028480 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:35.028483 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:35.029840 (+ 1357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:35.029841 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:35.031035 (+ 1194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:35.031037 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:35.031955 (+ 918us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:35.031956 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:35.032017 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:35.032021 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 15:15:35.032064 (+ 43us) client_negotiation.cc:240] Negotiation successful
0807 15:15:35.032104 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":299,"mutex_wait_us":68,"thread_start_us":75,"threads_started":1}
I20260807 15:15:35.034659 26717 tablet_service.cc:1467] Processing CreateTablet for tablet d79d80efce1b41ad8af1182477ff301d (DEFAULT_TABLE table=test-table [id=471f81670f2a416f9d4c0311a1c40513]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:15:35.035000 26717 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d79d80efce1b41ad8af1182477ff301d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:15:35.035898 27249 tablet_bootstrap.cc:492] T d79d80efce1b41ad8af1182477ff301d P 6f999502eed741cebab3c92e231b4483: Bootstrap starting.
I20260807 15:15:35.036563 27249 tablet_bootstrap.cc:654] T d79d80efce1b41ad8af1182477ff301d P 6f999502eed741cebab3c92e231b4483: Neither blocks nor log segments found. Creating new log.
I20260807 15:15:35.036577 27250 tablet_bootstrap.cc:492] T d79d80efce1b41ad8af1182477ff301d P bede7706385c4e17b2b9bafb10a0398f: Bootstrap starting.
I20260807 15:15:35.037158 27249 log.cc:824] T d79d80efce1b41ad8af1182477ff301d P 6f999502eed741cebab3c92e231b4483: Log is configured to *not* fsync() on all Append() calls
I20260807 15:15:35.037216 27250 tablet_bootstrap.cc:654] T d79d80efce1b41ad8af1182477ff301d P bede7706385c4e17b2b9bafb10a0398f: Neither blocks nor log segments found. Creating new log.
I20260807 15:15:35.037809 27250 log.cc:824] T d79d80efce1b41ad8af1182477ff301d P bede7706385c4e17b2b9bafb10a0398f: Log is configured to *not* fsync() on all Append() calls
I20260807 15:15:35.037979 27249 tablet_bootstrap.cc:492] T d79d80efce1b41ad8af1182477ff301d P 6f999502eed741cebab3c92e231b4483: No bootstrap required, opened a new log
I20260807 15:15:35.038065 27249 ts_tablet_manager.cc:1397] T d79d80efce1b41ad8af1182477ff301d P 6f999502eed741cebab3c92e231b4483: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260807 15:15:35.038343 27255 tablet_bootstrap.cc:492] T d79d80efce1b41ad8af1182477ff301d P 89d925f6e24b4f768e619cde5f376652: Bootstrap starting.
I20260807 15:15:35.038524 27250 tablet_bootstrap.cc:492] T d79d80efce1b41ad8af1182477ff301d P bede7706385c4e17b2b9bafb10a0398f: No bootstrap required, opened a new log
I20260807 15:15:35.038595 27250 ts_tablet_manager.cc:1397] T d79d80efce1b41ad8af1182477ff301d P bede7706385c4e17b2b9bafb10a0398f: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260807 15:15:35.038967 27255 tablet_bootstrap.cc:654] T d79d80efce1b41ad8af1182477ff301d P 89d925f6e24b4f768e619cde5f376652: Neither blocks nor log segments found. Creating new log.
I20260807 15:15:35.039525 27255 log.cc:824] T d79d80efce1b41ad8af1182477ff301d P 89d925f6e24b4f768e619cde5f376652: Log is configured to *not* fsync() on all Append() calls
I20260807 15:15:35.039626 27249 raft_consensus.cc:348] T d79d80efce1b41ad8af1182477ff301d P 6f999502eed741cebab3c92e231b4483 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bede7706385c4e17b2b9bafb10a0398f" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 38613 } } peers { permanent_uuid: "6f999502eed741cebab3c92e231b4483" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 46529 } } peers { permanent_uuid: "89d925f6e24b4f768e619cde5f376652" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 36245 } }
I20260807 15:15:35.039816 27249 raft_consensus.cc:374] T d79d80efce1b41ad8af1182477ff301d P 6f999502eed741cebab3c92e231b4483 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:15:35.039865 27249 raft_consensus.cc:729] T d79d80efce1b41ad8af1182477ff301d P 6f999502eed741cebab3c92e231b4483 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f999502eed741cebab3c92e231b4483, State: Initialized, Role: FOLLOWER
I20260807 15:15:35.039983 27249 consensus_queue.cc:260] T d79d80efce1b41ad8af1182477ff301d P 6f999502eed741cebab3c92e231b4483 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bede7706385c4e17b2b9bafb10a0398f" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 38613 } } peers { permanent_uuid: "6f999502eed741cebab3c92e231b4483" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 46529 } } peers { permanent_uuid: "89d925f6e24b4f768e619cde5f376652" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 36245 } }
I20260807 15:15:35.040273 27028 heartbeater.cc:503] Master 127.29.194.62:42211 was elected leader, sending a full tablet report...
I20260807 15:15:35.040257 27249 ts_tablet_manager.cc:1428] T d79d80efce1b41ad8af1182477ff301d P 6f999502eed741cebab3c92e231b4483: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260807 15:15:35.040534 27255 tablet_bootstrap.cc:492] T d79d80efce1b41ad8af1182477ff301d P 89d925f6e24b4f768e619cde5f376652: No bootstrap required, opened a new log
I20260807 15:15:35.040611 27255 ts_tablet_manager.cc:1397] T d79d80efce1b41ad8af1182477ff301d P 89d925f6e24b4f768e619cde5f376652: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260807 15:15:35.040542 27250 raft_consensus.cc:348] T d79d80efce1b41ad8af1182477ff301d P bede7706385c4e17b2b9bafb10a0398f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bede7706385c4e17b2b9bafb10a0398f" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 38613 } } peers { permanent_uuid: "6f999502eed741cebab3c92e231b4483" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 46529 } } peers { permanent_uuid: "89d925f6e24b4f768e619cde5f376652" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 36245 } }
I20260807 15:15:35.040701 27250 raft_consensus.cc:374] T d79d80efce1b41ad8af1182477ff301d P bede7706385c4e17b2b9bafb10a0398f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:15:35.040743 27250 raft_consensus.cc:729] T d79d80efce1b41ad8af1182477ff301d P bede7706385c4e17b2b9bafb10a0398f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bede7706385c4e17b2b9bafb10a0398f, State: Initialized, Role: FOLLOWER
I20260807 15:15:35.040846 27250 consensus_queue.cc:260] T d79d80efce1b41ad8af1182477ff301d P bede7706385c4e17b2b9bafb10a0398f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bede7706385c4e17b2b9bafb10a0398f" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 38613 } } peers { permanent_uuid: "6f999502eed741cebab3c92e231b4483" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 46529 } } peers { permanent_uuid: "89d925f6e24b4f768e619cde5f376652" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 36245 } }
I20260807 15:15:35.041138 27250 ts_tablet_manager.cc:1428] T d79d80efce1b41ad8af1182477ff301d P bede7706385c4e17b2b9bafb10a0398f: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260807 15:15:35.041373 27226 heartbeater.cc:503] Master 127.29.194.62:42211 was elected leader, sending a full tablet report...
I20260807 15:15:35.042448 27255 raft_consensus.cc:348] T d79d80efce1b41ad8af1182477ff301d P 89d925f6e24b4f768e619cde5f376652 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bede7706385c4e17b2b9bafb10a0398f" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 38613 } } peers { permanent_uuid: "6f999502eed741cebab3c92e231b4483" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 46529 } } peers { permanent_uuid: "89d925f6e24b4f768e619cde5f376652" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 36245 } }
I20260807 15:15:35.042578 27255 raft_consensus.cc:374] T d79d80efce1b41ad8af1182477ff301d P 89d925f6e24b4f768e619cde5f376652 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:15:35.042599 27255 raft_consensus.cc:729] T d79d80efce1b41ad8af1182477ff301d P 89d925f6e24b4f768e619cde5f376652 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 89d925f6e24b4f768e619cde5f376652, State: Initialized, Role: FOLLOWER
I20260807 15:15:35.042689 27255 consensus_queue.cc:260] T d79d80efce1b41ad8af1182477ff301d P 89d925f6e24b4f768e619cde5f376652 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bede7706385c4e17b2b9bafb10a0398f" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 38613 } } peers { permanent_uuid: "6f999502eed741cebab3c92e231b4483" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 46529 } } peers { permanent_uuid: "89d925f6e24b4f768e619cde5f376652" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 36245 } }
I20260807 15:15:35.042941 27255 ts_tablet_manager.cc:1428] T d79d80efce1b41ad8af1182477ff301d P 89d925f6e24b4f768e619cde5f376652: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260807 15:15:35.042999 26824 heartbeater.cc:503] Master 127.29.194.62:42211 was elected leader, sending a full tablet report...
W20260807 15:15:35.098285 27030 tablet.cc:2369] T d79d80efce1b41ad8af1182477ff301d P 6f999502eed741cebab3c92e231b4483: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 15:15:35.123488 27258 raft_consensus.cc:482] T d79d80efce1b41ad8af1182477ff301d P bede7706385c4e17b2b9bafb10a0398f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 15:15:35.123626 27258 raft_consensus.cc:504] T d79d80efce1b41ad8af1182477ff301d P bede7706385c4e17b2b9bafb10a0398f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bede7706385c4e17b2b9bafb10a0398f" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 38613 } } peers { permanent_uuid: "6f999502eed741cebab3c92e231b4483" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 46529 } } peers { permanent_uuid: "89d925f6e24b4f768e619cde5f376652" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 36245 } }
I20260807 15:15:35.123955 27258 leader_election.cc:288] T d79d80efce1b41ad8af1182477ff301d P bede7706385c4e17b2b9bafb10a0398f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f999502eed741cebab3c92e231b4483 (127.29.194.2:46529), 89d925f6e24b4f768e619cde5f376652 (127.29.194.1:36245)
I20260807 15:15:35.127878 27248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:35.124232 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.194.3:55317 (local address 127.29.194.2:46529)
0807 15:15:35.124280 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:35.124282 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:35.124291 (+ 9us) server_negotiation.cc:407] Connection header received
0807 15:15:35.124323 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:35.124323 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:35.124335 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:35.124399 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:15:35.124570 (+ 171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:35.125301 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:35.126843 (+ 1542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:35.127472 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:35.127543 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:35.127552 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0807 15:15:35.127755 (+ 203us) server_negotiation.cc:299] Negotiation successful
0807 15:15:35.127802 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260807 15:15:35.127878 27246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:35.124243 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.194.3:45347 (local address 127.29.194.1:36245)
0807 15:15:35.124285 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:35.124288 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:35.124470 (+ 182us) server_negotiation.cc:407] Connection header received
0807 15:15:35.124522 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:35.124522 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:35.124530 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:35.124575 (+ 45us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:15:35.124741 (+ 166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:35.125418 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:35.126843 (+ 1425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:35.127453 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:35.127543 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:35.127552 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0807 15:15:35.127755 (+ 203us) server_negotiation.cc:299] Negotiation successful
0807 15:15:35.127811 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260807 15:15:35.127946 27229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:35.124154 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.2:46529 (local address 127.29.194.3:55317)
0807 15:15:35.124216 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:35.124227 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 15:15:35.124268 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:15:35.124420 (+ 152us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:15:35.124421 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:15:35.124423 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:15:35.124503 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:35.124506 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:35.125396 (+ 890us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:35.125396 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:35.126732 (+ 1336us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:35.126735 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:35.127578 (+ 843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:35.127580 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:35.127677 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:35.127682 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 15:15:35.127747 (+ 65us) client_negotiation.cc:240] Negotiation successful
0807 15:15:35.127797 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43,"mutex_wait_us":11}
I20260807 15:15:35.128131 27262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:35.124169 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.1:36245 (local address 127.29.194.3:45347)
0807 15:15:35.124389 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:35.124398 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0807 15:15:35.124467 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:15:35.124607 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:15:35.124608 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:15:35.124609 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:15:35.124678 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:35.124679 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:35.125563 (+ 884us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:35.125564 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:35.126732 (+ 1168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:35.126735 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:35.127579 (+ 844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:35.127580 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:35.127677 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:35.127682 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 15:15:35.127747 (+ 65us) client_negotiation.cc:240] Negotiation successful
0807 15:15:35.127797 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":204,"mutex_wait_us":13,"thread_start_us":122,"threads_started":1}
I20260807 15:15:35.128345 26967 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d79d80efce1b41ad8af1182477ff301d" candidate_uuid: "bede7706385c4e17b2b9bafb10a0398f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f999502eed741cebab3c92e231b4483" is_pre_election: true
I20260807 15:15:35.128546 26755 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d79d80efce1b41ad8af1182477ff301d" candidate_uuid: "bede7706385c4e17b2b9bafb10a0398f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "89d925f6e24b4f768e619cde5f376652" is_pre_election: true
I20260807 15:15:35.128600 26967 raft_consensus.cc:2454] T d79d80efce1b41ad8af1182477ff301d P 6f999502eed741cebab3c92e231b4483 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bede7706385c4e17b2b9bafb10a0398f in term 0.
I20260807 15:15:35.128700 26755 raft_consensus.cc:2454] T d79d80efce1b41ad8af1182477ff301d P 89d925f6e24b4f768e619cde5f376652 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bede7706385c4e17b2b9bafb10a0398f in term 0.
I20260807 15:15:35.128887 27057 leader_election.cc:302] T d79d80efce1b41ad8af1182477ff301d P bede7706385c4e17b2b9bafb10a0398f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f999502eed741cebab3c92e231b4483, bede7706385c4e17b2b9bafb10a0398f; no voters:
I20260807 15:15:35.129053 27258 raft_consensus.cc:2781] T d79d80efce1b41ad8af1182477ff301d P bede7706385c4e17b2b9bafb10a0398f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 15:15:35.129122 27258 raft_consensus.cc:482] T d79d80efce1b41ad8af1182477ff301d P bede7706385c4e17b2b9bafb10a0398f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 15:15:35.129137 27258 raft_consensus.cc:3037] T d79d80efce1b41ad8af1182477ff301d P bede7706385c4e17b2b9bafb10a0398f [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:15:35.129933 27258 raft_consensus.cc:504] T d79d80efce1b41ad8af1182477ff301d P bede7706385c4e17b2b9bafb10a0398f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bede7706385c4e17b2b9bafb10a0398f" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 38613 } } peers { permanent_uuid: "6f999502eed741cebab3c92e231b4483" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 46529 } } peers { permanent_uuid: "89d925f6e24b4f768e619cde5f376652" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 36245 } }
I20260807 15:15:35.130075 27258 leader_election.cc:288] T d79d80efce1b41ad8af1182477ff301d P bede7706385c4e17b2b9bafb10a0398f [CANDIDATE]: Term 1 election: Requested vote from peers 6f999502eed741cebab3c92e231b4483 (127.29.194.2:46529), 89d925f6e24b4f768e619cde5f376652 (127.29.194.1:36245)
I20260807 15:15:35.130249 26755 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d79d80efce1b41ad8af1182477ff301d" candidate_uuid: "bede7706385c4e17b2b9bafb10a0398f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "89d925f6e24b4f768e619cde5f376652"
I20260807 15:15:35.130340 26755 raft_consensus.cc:3037] T d79d80efce1b41ad8af1182477ff301d P 89d925f6e24b4f768e619cde5f376652 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:15:35.131069 26755 raft_consensus.cc:2454] T d79d80efce1b41ad8af1182477ff301d P 89d925f6e24b4f768e619cde5f376652 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bede7706385c4e17b2b9bafb10a0398f in term 1.
I20260807 15:15:35.131191 26967 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d79d80efce1b41ad8af1182477ff301d" candidate_uuid: "bede7706385c4e17b2b9bafb10a0398f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f999502eed741cebab3c92e231b4483"
I20260807 15:15:35.131263 27061 leader_election.cc:302] T d79d80efce1b41ad8af1182477ff301d P bede7706385c4e17b2b9bafb10a0398f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 89d925f6e24b4f768e619cde5f376652, bede7706385c4e17b2b9bafb10a0398f; no voters:
I20260807 15:15:35.131281 26967 raft_consensus.cc:3037] T d79d80efce1b41ad8af1182477ff301d P 6f999502eed741cebab3c92e231b4483 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:15:35.131385 27258 raft_consensus.cc:2781] T d79d80efce1b41ad8af1182477ff301d P bede7706385c4e17b2b9bafb10a0398f [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:15:35.131572 27258 raft_consensus.cc:686] T d79d80efce1b41ad8af1182477ff301d P bede7706385c4e17b2b9bafb10a0398f [term 1 LEADER]: Becoming Leader. State: Replica: bede7706385c4e17b2b9bafb10a0398f, State: Running, Role: LEADER
I20260807 15:15:35.131667 27258 consensus_queue.cc:237] T d79d80efce1b41ad8af1182477ff301d P bede7706385c4e17b2b9bafb10a0398f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bede7706385c4e17b2b9bafb10a0398f" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 38613 } } peers { permanent_uuid: "6f999502eed741cebab3c92e231b4483" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 46529 } } peers { permanent_uuid: "89d925f6e24b4f768e619cde5f376652" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 36245 } }
I20260807 15:15:35.132144 26967 raft_consensus.cc:2454] T d79d80efce1b41ad8af1182477ff301d P 6f999502eed741cebab3c92e231b4483 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bede7706385c4e17b2b9bafb10a0398f in term 1.
I20260807 15:15:35.132490 26532 catalog_manager.cc:5495] T d79d80efce1b41ad8af1182477ff301d P bede7706385c4e17b2b9bafb10a0398f reported cstate change: term changed from 0 to 1, leader changed from <none> to bede7706385c4e17b2b9bafb10a0398f (127.29.194.3). New cstate: current_term: 1 leader_uuid: "bede7706385c4e17b2b9bafb10a0398f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bede7706385c4e17b2b9bafb10a0398f" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 38613 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6f999502eed741cebab3c92e231b4483" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 46529 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "89d925f6e24b4f768e619cde5f376652" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 36245 } health_report { overall_health: UNKNOWN } } }
I20260807 15:15:35.150272 27245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:35.147779 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55770 (local address 127.29.194.3:38613)
0807 15:15:35.147862 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:35.147864 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:35.147874 (+ 10us) server_negotiation.cc:407] Connection header received
0807 15:15:35.147917 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:35.147918 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:35.147937 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:35.148001 (+ 64us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 15:15:35.148137 (+ 136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:35.148632 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:35.149383 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:35.149627 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:35.149688 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:35.149827 (+ 139us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 15:15:35.149938 (+ 111us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 15:15:35.150013 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0807 15:15:35.150099 (+ 86us) server_negotiation.cc:299] Negotiation successful
0807 15:15:35.150155 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260807 15:15:35.154441 26967 raft_consensus.cc:1261] T d79d80efce1b41ad8af1182477ff301d P 6f999502eed741cebab3c92e231b4483 [term 1 FOLLOWER]: Refusing update from remote peer bede7706385c4e17b2b9bafb10a0398f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:15:35.154435 26755 raft_consensus.cc:1261] T d79d80efce1b41ad8af1182477ff301d P 89d925f6e24b4f768e619cde5f376652 [term 1 FOLLOWER]: Refusing update from remote peer bede7706385c4e17b2b9bafb10a0398f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:15:35.154835 27264 consensus_queue.cc:1035] T d79d80efce1b41ad8af1182477ff301d P bede7706385c4e17b2b9bafb10a0398f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6f999502eed741cebab3c92e231b4483" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 46529 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:15:35.155072 27264 consensus_queue.cc:1035] T d79d80efce1b41ad8af1182477ff301d P bede7706385c4e17b2b9bafb10a0398f [LEADER]: Connected to new peer: Peer: permanent_uuid: "89d925f6e24b4f768e619cde5f376652" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 36245 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:15:35.157382 27268 mvcc.cc:205] Tried to move back new op lower bound from 7315930051188375552 to 7315930051100041216. Current Snapshot: MvccSnapshot[applied={T|T < 7315930051188375552}]
I20260807 15:15:35.157819 27270 mvcc.cc:205] Tried to move back new op lower bound from 7315930051188375552 to 7315930051100041216. Current Snapshot: MvccSnapshot[applied={T|T < 7315930051188375552}]
I20260807 15:15:35.160364 26530 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41586:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 15:15:35.160470 26530 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41586:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 15:15:35.161223 26530 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ffdbfd49a4734e86af35d0ab8ae866fc: Sending DeleteTablet for 3 replicas of tablet d79d80efce1b41ad8af1182477ff301d
I20260807 15:15:35.161583 27132 tablet_service.cc:1514] Processing DeleteTablet for tablet d79d80efce1b41ad8af1182477ff301d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:15:35 UTC) from {username='kudu', principal='kudu/127.29.194.62@KRBTEST.COM'} at 127.0.0.1:55766
I20260807 15:15:35.161659 26933 tablet_service.cc:1514] Processing DeleteTablet for tablet d79d80efce1b41ad8af1182477ff301d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:15:35 UTC) from {username='kudu', principal='kudu/127.29.194.62@KRBTEST.COM'} at 127.0.0.1:50298
I20260807 15:15:35.161689 26717 tablet_service.cc:1514] Processing DeleteTablet for tablet d79d80efce1b41ad8af1182477ff301d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:15:35 UTC) from {username='kudu', principal='kudu/127.29.194.62@KRBTEST.COM'} at 127.0.0.1:53192
I20260807 15:15:35.161875 27278 tablet_replica.cc:330] T d79d80efce1b41ad8af1182477ff301d P bede7706385c4e17b2b9bafb10a0398f: stopping tablet replica
I20260807 15:15:35.161984 27278 raft_consensus.cc:2229] T d79d80efce1b41ad8af1182477ff301d P bede7706385c4e17b2b9bafb10a0398f [term 1 LEADER]: Raft consensus shutting down.
I20260807 15:15:35.162084 27278 raft_consensus.cc:2258] T d79d80efce1b41ad8af1182477ff301d P bede7706385c4e17b2b9bafb10a0398f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:15:35.162168 30472 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7fZSIL/build/release/bin/kudu with pid 26632
I20260807 15:15:35.165019 27278 ts_tablet_manager.cc:1905] T d79d80efce1b41ad8af1182477ff301d P bede7706385c4e17b2b9bafb10a0398f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:15:35.167003 27278 ts_tablet_manager.cc:1918] T d79d80efce1b41ad8af1182477ff301d P bede7706385c4e17b2b9bafb10a0398f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 15:15:35.167101 27278 log.cc:1196] T d79d80efce1b41ad8af1182477ff301d P bede7706385c4e17b2b9bafb10a0398f: Deleting WAL directory at /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115480851507-30472-0/minicluster-data/ts-2/wal/wals/d79d80efce1b41ad8af1182477ff301d
I20260807 15:15:35.167467 27278 ts_tablet_manager.cc:1939] T d79d80efce1b41ad8af1182477ff301d P bede7706385c4e17b2b9bafb10a0398f: Deleting consensus metadata
I20260807 15:15:35.168059 26508 catalog_manager.cc:4841] TS bede7706385c4e17b2b9bafb10a0398f (127.29.194.3:38613): tablet d79d80efce1b41ad8af1182477ff301d (table test-table [id=471f81670f2a416f9d4c0311a1c40513]) successfully deleted
W20260807 15:15:35.175926 26511 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.29.194.1:36245 (error 108)
I20260807 15:15:35.176005 27279 tablet_replica.cc:330] T d79d80efce1b41ad8af1182477ff301d P 6f999502eed741cebab3c92e231b4483: stopping tablet replica
I20260807 15:15:35.176215 27279 raft_consensus.cc:2229] T d79d80efce1b41ad8af1182477ff301d P 6f999502eed741cebab3c92e231b4483 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 15:15:35.176453 27279 raft_consensus.cc:2258] T d79d80efce1b41ad8af1182477ff301d P 6f999502eed741cebab3c92e231b4483 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:15:35.176606 30472 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7fZSIL/build/release/bin/kudu with pid 26829
I20260807 15:15:35.199608 27247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:35.199420 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.1:36245 (local address 127.0.0.1:53204)
0807 15:15:35.199495 (+ 75us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:35.199556 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.194.1:36245: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260807 15:15:35.199774 26511 catalog_manager.cc:4584] TS 89d925f6e24b4f768e619cde5f376652 (127.29.194.1:36245): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet d79d80efce1b41ad8af1182477ff301d: Network error: Client connection negotiation failed: client connection to 127.29.194.1:36245: connect: Connection refused (error 111)
I20260807 15:15:35.205024 30472 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7fZSIL/build/release/bin/kudu with pid 27037
I20260807 15:15:35.214687 27247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:35.214550 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.2:46529 (local address 127.0.0.1:50302)
0807 15:15:35.214613 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:35.214642 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.194.2:46529: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260807 15:15:35.219663 30472 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7fZSIL/build/release/bin/kudu with pid 26486
2026-08-07T15:15:35Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (2167 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-07T15:15:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T15:15:35Z Disabled control of system clock
I20260807 15:15:35.259295 30472 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7fZSIL/build/release/bin/kudu
/tmp/dist-test-task7fZSIL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115480851507-30472-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115480851507-30472-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115480851507-30472-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115480851507-30472-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.194.62:36371
--webserver_interface=127.29.194.62
--webserver_port=0
--builtin_ntp_servers=127.29.194.20:37107
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.194.62:36371
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260807 15:15:35.341585 27286 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:15:35.341835 27286 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:15:35.341857 27286 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:15:35.343088 27286 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:15:35.343147 27286 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:15:35.343159 27286 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:15:35.343173 27286 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:15:35.343183 27286 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:15:35.344632 27286 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.194.20:37107
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115480851507-30472-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115480851507-30472-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.194.62:36371
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.194.62:36371
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115480851507-30472-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.194.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27286
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115480851507-30472-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:15:35.344915 27286 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:15:35.345224 27286 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 15:15:35.350659 27286 server_base.cc:1034] running on GCE node
W20260807 15:15:35.350646 27296 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:15:35.350649 27291 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:15:35.350833 27292 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:15:35.351222 27286 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:15:35.351625 27286 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:15:35.352818 27286 hybrid_clock.cc:648] HybridClock initialized: now 1786115735352791 us; error 44 us; skew 500 ppm
I20260807 15:15:35.354625 27286 webserver.cc:460] Webserver started at http://127.29.194.62:39967/ using document root <none> and password file <none>
I20260807 15:15:35.354879 27286 fs_manager.cc:359] Metadata directory not provided
I20260807 15:15:35.354933 27286 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:15:35.355077 27286 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:15:35.356227 27286 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115480851507-30472-0/minicluster-data/master-0/data/instance:
uuid: "e766bca9bc9a4dd695fb50d9e947ecde"
format_stamp: "Formatted at 2026-08-07 15:15:35 on dist-test-slave-42z9"
server_key: "955c44ff14a803796d976de26f6e3993"
server_key_iv: "d5d6035573d8b6ebda1756097f0d036a"
server_key_version: "encryptionkey@0"
I20260807 15:15:35.356581 27286 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115480851507-30472-0/minicluster-data/master-0/wal/instance:
uuid: "e766bca9bc9a4dd695fb50d9e947ecde"
format_stamp: "Formatted at 2026-08-07 15:15:35 on dist-test-slave-42z9"
server_key: "955c44ff14a803796d976de26f6e3993"
server_key_iv: "d5d6035573d8b6ebda1756097f0d036a"
server_key_version: "encryptionkey@0"
I20260807 15:15:35.358279 27286 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.001s
I20260807 15:15:35.359233 27303 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:15:35.359601 27286 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:15:35.359722 27286 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115480851507-30472-0/minicluster-data/master-0/data,/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115480851507-30472-0/minicluster-data/master-0/wal
uuid: "e766bca9bc9a4dd695fb50d9e947ecde"
format_stamp: "Formatted at 2026-08-07 15:15:35 on dist-test-slave-42z9"
server_key: "955c44ff14a803796d976de26f6e3993"
server_key_iv: "d5d6035573d8b6ebda1756097f0d036a"
server_key_version: "encryptionkey@0"
I20260807 15:15:35.359802 27286 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115480851507-30472-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115480851507-30472-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115480851507-30472-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:15:35.371476 27286 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:15:35.371834 27286 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:15:35.371971 27286 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:15:35.375973 27286 rpc_server.cc:307] RPC server started. Bound to: 127.29.194.62:36371
I20260807 15:15:35.376010 27376 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.194.62:36371 every 8 connection(s)
I20260807 15:15:35.376336 27286 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115480851507-30472-0/minicluster-data/master-0/data/info.pb
I20260807 15:15:35.377338 27378 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:15:35.379848 27378 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e766bca9bc9a4dd695fb50d9e947ecde: Bootstrap starting.
I20260807 15:15:35.380492 27378 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e766bca9bc9a4dd695fb50d9e947ecde: Neither blocks nor log segments found. Creating new log.
I20260807 15:15:35.380781 27378 log.cc:824] T 00000000000000000000000000000000 P e766bca9bc9a4dd695fb50d9e947ecde: Log is configured to *not* fsync() on all Append() calls
I20260807 15:15:35.381536 27378 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e766bca9bc9a4dd695fb50d9e947ecde: No bootstrap required, opened a new log
I20260807 15:15:35.383664 27378 raft_consensus.cc:348] T 00000000000000000000000000000000 P e766bca9bc9a4dd695fb50d9e947ecde [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e766bca9bc9a4dd695fb50d9e947ecde" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 36371 } }
I20260807 15:15:35.383831 27378 raft_consensus.cc:374] T 00000000000000000000000000000000 P e766bca9bc9a4dd695fb50d9e947ecde [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:15:35.383854 27378 raft_consensus.cc:729] T 00000000000000000000000000000000 P e766bca9bc9a4dd695fb50d9e947ecde [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e766bca9bc9a4dd695fb50d9e947ecde, State: Initialized, Role: FOLLOWER
I20260807 15:15:35.383981 27378 consensus_queue.cc:260] T 00000000000000000000000000000000 P e766bca9bc9a4dd695fb50d9e947ecde [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e766bca9bc9a4dd695fb50d9e947ecde" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 36371 } }
I20260807 15:15:35.384063 27378 raft_consensus.cc:388] T 00000000000000000000000000000000 P e766bca9bc9a4dd695fb50d9e947ecde [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:15:35.384087 27378 raft_consensus.cc:482] T 00000000000000000000000000000000 P e766bca9bc9a4dd695fb50d9e947ecde [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:15:35.384116 27378 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e766bca9bc9a4dd695fb50d9e947ecde [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:15:35.384670 27378 raft_consensus.cc:504] T 00000000000000000000000000000000 P e766bca9bc9a4dd695fb50d9e947ecde [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e766bca9bc9a4dd695fb50d9e947ecde" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 36371 } }
I20260807 15:15:35.384771 27378 leader_election.cc:302] T 00000000000000000000000000000000 P e766bca9bc9a4dd695fb50d9e947ecde [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e766bca9bc9a4dd695fb50d9e947ecde; no voters:
I20260807 15:15:35.384886 30472 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7fZSIL/build/release/bin/kudu as pid 27286
I20260807 15:15:35.384971 27378 leader_election.cc:288] T 00000000000000000000000000000000 P e766bca9bc9a4dd695fb50d9e947ecde [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 15:15:35.385003 30472 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115480851507-30472-0/minicluster-data/master-0/wal/instance
I20260807 15:15:35.385040 27385 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e766bca9bc9a4dd695fb50d9e947ecde [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:15:35.385224 27385 raft_consensus.cc:686] T 00000000000000000000000000000000 P e766bca9bc9a4dd695fb50d9e947ecde [term 1 LEADER]: Becoming Leader. State: Replica: e766bca9bc9a4dd695fb50d9e947ecde, State: Running, Role: LEADER
I20260807 15:15:35.385221 30472 external_mini_cluster.cc:1428] Setting key bf766ed53e82295347bd47c8454413b9
I20260807 15:15:35.385262 27378 sys_catalog.cc:564] T 00000000000000000000000000000000 P e766bca9bc9a4dd695fb50d9e947ecde [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:15:35.385352 27385 consensus_queue.cc:237] T 00000000000000000000000000000000 P e766bca9bc9a4dd695fb50d9e947ecde [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e766bca9bc9a4dd695fb50d9e947ecde" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 36371 } }
I20260807 15:15:35.385921 27386 sys_catalog.cc:455] T 00000000000000000000000000000000 P e766bca9bc9a4dd695fb50d9e947ecde [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e766bca9bc9a4dd695fb50d9e947ecde" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e766bca9bc9a4dd695fb50d9e947ecde" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 36371 } } }
I20260807 15:15:35.386010 27386 sys_catalog.cc:458] T 00000000000000000000000000000000 P e766bca9bc9a4dd695fb50d9e947ecde [sys.catalog]: This master's current role is: LEADER
I20260807 15:15:35.386170 27390 sys_catalog.cc:455] T 00000000000000000000000000000000 P e766bca9bc9a4dd695fb50d9e947ecde [sys.catalog]: SysCatalogTable state changed. Reason: New leader e766bca9bc9a4dd695fb50d9e947ecde. Latest consensus state: current_term: 1 leader_uuid: "e766bca9bc9a4dd695fb50d9e947ecde" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e766bca9bc9a4dd695fb50d9e947ecde" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 36371 } } }
I20260807 15:15:35.386227 27390 sys_catalog.cc:458] T 00000000000000000000000000000000 P e766bca9bc9a4dd695fb50d9e947ecde [sys.catalog]: This master's current role is: LEADER
I20260807 15:15:35.386380 27403 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:15:35.386929 27403 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:15:35.388696 27403 catalog_manager.cc:1347] Generated new cluster ID: 2a306465fe10440b8ec6eb0880872d86
I20260807 15:15:35.388746 27403 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:15:35.391450 27393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:35.385749 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36476 (local address 127.29.194.62:36371)
0807 15:15:35.386083 (+ 334us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:35.386092 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:35.386128 (+ 36us) server_negotiation.cc:407] Connection header received
0807 15:15:35.386248 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:35.386256 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:35.386330 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:35.386705 (+ 375us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:15:35.386800 (+ 95us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:35.388079 (+ 1279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:35.389173 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:35.389796 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:35.389910 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:35.390657 (+ 747us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:15:35.390660 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:15:35.390661 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:15:35.390863 (+ 202us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:15:35.390902 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:15:35.390916 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:15:35.390971 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0807 15:15:35.391061 (+ 90us) server_negotiation.cc:299] Negotiation successful
0807 15:15:35.391238 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":252,"thread_start_us":77,"threads_started":1}
I20260807 15:15:35.393239 27403 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:15:35.393782 27403 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:15:35.404040 27403 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e766bca9bc9a4dd695fb50d9e947ecde: Generated new TSK 0
I20260807 15:15:35.404233 27403 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 15:15:35.408536 30472 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7fZSIL/build/release/bin/kudu
/tmp/dist-test-task7fZSIL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115480851507-30472-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115480851507-30472-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115480851507-30472-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115480851507-30472-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.194.1:0
--local_ip_for_outbound_sockets=127.29.194.1
--webserver_interface=127.29.194.1
--webserver_port=0
--tserver_master_addrs=127.29.194.62:36371
--builtin_ntp_servers=127.29.194.20:37107
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260807 15:15:35.495949 27421 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:15:35.496189 27421 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:15:35.496222 27421 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:15:35.497427 27421 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:15:35.497500 27421 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:15:35.497545 27421 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.194.1
I20260807 15:15:35.498980 27421 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.194.20:37107
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115480851507-30472-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115480851507-30472-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.194.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115480851507-30472-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.194.1
--webserver_port=0
--tserver_master_addrs=127.29.194.62:36371
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27421
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.194.1
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115480851507-30472-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:15:35.499234 27421 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:15:35.499949 27421 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:15:35.506800 27428 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:15:35.506793 27429 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:15:35.506788 27433 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:15:35.507025 27421 server_base.cc:1034] running on GCE node
I20260807 15:15:35.507339 27421 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:15:35.507654 27421 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:15:35.508838 27421 hybrid_clock.cc:648] HybridClock initialized: now 1786115735508823 us; error 31 us; skew 500 ppm
I20260807 15:15:35.510447 27421 webserver.cc:460] Webserver started at http://127.29.194.1:36899/ using document root <none> and password file <none>
I20260807 15:15:35.510725 27421 fs_manager.cc:359] Metadata directory not provided
I20260807 15:15:35.510785 27421 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:15:35.510921 27421 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:15:35.512487 27421 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115480851507-30472-0/minicluster-data/ts-0/data/instance:
uuid: "2c749b22952e480e8b786b3287e678e6"
format_stamp: "Formatted at 2026-08-07 15:15:35 on dist-test-slave-42z9"
server_key: "63d15013d0164ed321b8ddd3d18ddffd"
server_key_iv: "6002665d439e69bb6b1a23f0c7bd3dcf"
server_key_version: "encryptionkey@0"
I20260807 15:15:35.512946 27421 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115480851507-30472-0/minicluster-data/ts-0/wal/instance:
uuid: "2c749b22952e480e8b786b3287e678e6"
format_stamp: "Formatted at 2026-08-07 15:15:35 on dist-test-slave-42z9"
server_key: "63d15013d0164ed321b8ddd3d18ddffd"
server_key_iv: "6002665d439e69bb6b1a23f0c7bd3dcf"
server_key_version: "encryptionkey@0"
I20260807 15:15:35.514763 27421 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.004s sys 0.000s
I20260807 15:15:35.515847 27441 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:15:35.516125 27421 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:15:35.516240 27421 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115480851507-30472-0/minicluster-data/ts-0/data,/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115480851507-30472-0/minicluster-data/ts-0/wal
uuid: "2c749b22952e480e8b786b3287e678e6"
format_stamp: "Formatted at 2026-08-07 15:15:35 on dist-test-slave-42z9"
server_key: "63d15013d0164ed321b8ddd3d18ddffd"
server_key_iv: "6002665d439e69bb6b1a23f0c7bd3dcf"
server_key_version: "encryptionkey@0"
I20260807 15:15:35.516323 27421 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115480851507-30472-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115480851507-30472-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115480851507-30472-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:15:35.549830 27421 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:15:35.550154 27421 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:15:35.550269 27421 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:15:35.550495 27421 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:15:35.550881 27421 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:15:35.550915 27421 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:15:35.550937 27421 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:15:35.550951 27421 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:15:35.556855 27421 rpc_server.cc:307] RPC server started. Bound to: 127.29.194.1:44623
I20260807 15:15:35.556911 27607 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.194.1:44623 every 8 connection(s)
I20260807 15:15:35.557256 27421 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115480851507-30472-0/minicluster-data/ts-0/data/info.pb
I20260807 15:15:35.562660 27393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:35.558076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.194.1:38383 (local address 127.29.194.62:36371)
0807 15:15:35.558131 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:35.558133 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:35.558400 (+ 267us) server_negotiation.cc:407] Connection header received
0807 15:15:35.558934 (+ 534us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:35.558935 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:35.558950 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:35.559008 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:15:35.559719 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:35.560240 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:35.561218 (+ 978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:35.561418 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:35.561517 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:35.562353 (+ 836us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:15:35.562354 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:15:35.562354 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:15:35.562359 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:15:35.562374 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:15:35.562379 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:15:35.562423 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0807 15:15:35.562530 (+ 107us) server_negotiation.cc:299] Negotiation successful
0807 15:15:35.562571 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260807 15:15:35.562901 27611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:35.558050 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:36371 (local address 127.29.194.1:38383)
0807 15:15:35.558279 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:35.558317 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0807 15:15:35.558826 (+ 509us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:15:35.559082 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:15:35.559085 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:15:35.559094 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:15:35.559629 (+ 535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:35.559634 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:35.560335 (+ 701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:35.560336 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:35.561130 (+ 794us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:35.561133 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:35.561539 (+ 406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:35.561540 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:35.561669 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:35.562211 (+ 542us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 15:15:35.562212 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:15:35.562243 (+ 31us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 15:15:35.562269 (+ 26us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:15:35.562436 (+ 167us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:15:35.562442 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:15:35.562445 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 15:15:35.562532 (+ 87us) client_negotiation.cc:240] Negotiation successful
0807 15:15:35.562643 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":95,"threads_started":1}
I20260807 15:15:35.563720 27608 heartbeater.cc:347] Connected to a master server at 127.29.194.62:36371
I20260807 15:15:35.563823 27608 heartbeater.cc:464] Registering TS with master...
I20260807 15:15:35.564023 27608 heartbeater.cc:511] Master 127.29.194.62:36371 requested a full tablet report, sending...
I20260807 15:15:35.564776 27327 ts_manager.cc:194] Registered new tserver with Master: 2c749b22952e480e8b786b3287e678e6 (127.29.194.1:44623)
I20260807 15:15:35.565280 30472 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7fZSIL/build/release/bin/kudu as pid 27421
I20260807 15:15:35.565392 30472 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115480851507-30472-0/minicluster-data/ts-0/wal/instance
I20260807 15:15:35.565522 27327 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.29.194.1:38383
I20260807 15:15:35.565603 30472 external_mini_cluster.cc:1428] Setting key 49fb7a39fa3c64f90b92f7f9fba7f5d7
I20260807 15:15:35.566509 30472 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7fZSIL/build/release/bin/kudu
/tmp/dist-test-task7fZSIL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115480851507-30472-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115480851507-30472-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115480851507-30472-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115480851507-30472-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.194.2:0
--local_ip_for_outbound_sockets=127.29.194.2
--webserver_interface=127.29.194.2
--webserver_port=0
--tserver_master_addrs=127.29.194.62:36371
--builtin_ntp_servers=127.29.194.20:37107
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260807 15:15:35.648543 27612 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:15:35.648752 27612 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:15:35.648773 27612 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:15:35.649981 27612 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:15:35.650028 27612 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:15:35.650054 27612 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.194.2
I20260807 15:15:35.651481 27612 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.194.20:37107
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115480851507-30472-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115480851507-30472-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.194.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115480851507-30472-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.194.2
--webserver_port=0
--tserver_master_addrs=127.29.194.62:36371
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27612
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.194.2
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115480851507-30472-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:15:35.651737 27612 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:15:35.652030 27612 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:15:35.656985 27619 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:15:35.657133 27612 server_base.cc:1034] running on GCE node
W20260807 15:15:35.656985 27625 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:15:35.656978 27620 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:15:35.657383 27612 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:15:35.657637 27612 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:15:35.658783 27612 hybrid_clock.cc:648] HybridClock initialized: now 1786115735658768 us; error 30 us; skew 500 ppm
I20260807 15:15:35.660038 27612 webserver.cc:460] Webserver started at http://127.29.194.2:39223/ using document root <none> and password file <none>
I20260807 15:15:35.660269 27612 fs_manager.cc:359] Metadata directory not provided
I20260807 15:15:35.660317 27612 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:15:35.660450 27612 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:15:35.661546 27612 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115480851507-30472-0/minicluster-data/ts-1/data/instance:
uuid: "5e5d467cacee4c26b6ae9ca20f8be7ed"
format_stamp: "Formatted at 2026-08-07 15:15:35 on dist-test-slave-42z9"
server_key: "41550a2de5d98b09ef7823f612442c5b"
server_key_iv: "d91f82cf70fbb3674b7e760a151ffd87"
server_key_version: "encryptionkey@0"
I20260807 15:15:35.661880 27612 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115480851507-30472-0/minicluster-data/ts-1/wal/instance:
uuid: "5e5d467cacee4c26b6ae9ca20f8be7ed"
format_stamp: "Formatted at 2026-08-07 15:15:35 on dist-test-slave-42z9"
server_key: "41550a2de5d98b09ef7823f612442c5b"
server_key_iv: "d91f82cf70fbb3674b7e760a151ffd87"
server_key_version: "encryptionkey@0"
I20260807 15:15:35.663188 27612 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:15:35.663903 27634 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:15:35.664223 27612 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:15:35.664398 27612 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115480851507-30472-0/minicluster-data/ts-1/data,/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115480851507-30472-0/minicluster-data/ts-1/wal
uuid: "5e5d467cacee4c26b6ae9ca20f8be7ed"
format_stamp: "Formatted at 2026-08-07 15:15:35 on dist-test-slave-42z9"
server_key: "41550a2de5d98b09ef7823f612442c5b"
server_key_iv: "d91f82cf70fbb3674b7e760a151ffd87"
server_key_version: "encryptionkey@0"
I20260807 15:15:35.664506 27612 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115480851507-30472-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115480851507-30472-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115480851507-30472-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:15:35.675807 27612 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:15:35.676136 27612 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:15:35.676270 27612 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:15:35.676493 27612 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:15:35.676821 27612 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:15:35.676872 27612 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:15:35.676905 27612 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:15:35.676929 27612 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:15:35.682999 27612 rpc_server.cc:307] RPC server started. Bound to: 127.29.194.2:34341
I20260807 15:15:35.683074 27798 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.194.2:34341 every 8 connection(s)
I20260807 15:15:35.683387 27612 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115480851507-30472-0/minicluster-data/ts-1/data/info.pb
I20260807 15:15:35.688616 27393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:35.684137 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.194.2:54363 (local address 127.29.194.62:36371)
0807 15:15:35.684191 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:35.684193 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:35.684406 (+ 213us) server_negotiation.cc:407] Connection header received
0807 15:15:35.684948 (+ 542us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:35.684949 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:35.684961 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:35.685010 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:15:35.685823 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:35.686328 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:35.687319 (+ 991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:35.687510 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:35.687567 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:35.688355 (+ 788us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:15:35.688356 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:15:35.688357 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:15:35.688362 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:15:35.688376 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:15:35.688380 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:15:35.688422 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0807 15:15:35.688500 (+ 78us) server_negotiation.cc:299] Negotiation successful
0807 15:15:35.688530 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260807 15:15:35.688855 27803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:35.684107 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:36371 (local address 127.29.194.2:54363)
0807 15:15:35.684285 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:35.684321 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0807 15:15:35.684832 (+ 511us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:15:35.685091 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:15:35.685095 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:15:35.685103 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:15:35.685734 (+ 631us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:35.685739 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:35.686419 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:35.686420 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:35.687233 (+ 813us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:35.687235 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:35.687587 (+ 352us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:35.687588 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:35.687713 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:35.688222 (+ 509us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 15:15:35.688222 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:15:35.688251 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 15:15:35.688274 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:15:35.688429 (+ 155us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:15:35.688434 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:15:35.688438 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 15:15:35.688497 (+ 59us) client_negotiation.cc:240] Negotiation successful
0807 15:15:35.688626 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":75,"threads_started":1}
I20260807 15:15:35.689421 27799 heartbeater.cc:347] Connected to a master server at 127.29.194.62:36371
I20260807 15:15:35.689565 27799 heartbeater.cc:464] Registering TS with master...
I20260807 15:15:35.689751 27799 heartbeater.cc:511] Master 127.29.194.62:36371 requested a full tablet report, sending...
I20260807 15:15:35.690093 27327 ts_manager.cc:194] Registered new tserver with Master: 5e5d467cacee4c26b6ae9ca20f8be7ed (127.29.194.2:34341)
I20260807 15:15:35.690481 27327 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.29.194.2:54363
I20260807 15:15:35.692412 30472 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7fZSIL/build/release/bin/kudu as pid 27612
I20260807 15:15:35.692540 30472 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115480851507-30472-0/minicluster-data/ts-1/wal/instance
I20260807 15:15:35.692775 30472 external_mini_cluster.cc:1428] Setting key 6b7f2007cff3a123c55209dc386e0671
I20260807 15:15:35.693812 30472 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7fZSIL/build/release/bin/kudu
/tmp/dist-test-task7fZSIL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115480851507-30472-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115480851507-30472-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115480851507-30472-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115480851507-30472-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.194.3:0
--local_ip_for_outbound_sockets=127.29.194.3
--webserver_interface=127.29.194.3
--webserver_port=0
--tserver_master_addrs=127.29.194.62:36371
--builtin_ntp_servers=127.29.194.20:37107
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260807 15:15:35.776572 27804 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:15:35.776783 27804 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:15:35.776804 27804 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:15:35.777982 27804 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:15:35.778026 27804 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:15:35.778051 27804 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.194.3
I20260807 15:15:35.779423 27804 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.194.20:37107
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115480851507-30472-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115480851507-30472-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.194.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115480851507-30472-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.194.3
--webserver_port=0
--tserver_master_addrs=127.29.194.62:36371
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27804
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.194.3
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115480851507-30472-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:15:35.779657 27804 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:15:35.779963 27804 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:15:35.785159 27809 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:15:35.785208 27804 server_base.cc:1034] running on GCE node
W20260807 15:15:35.785147 27811 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:15:35.785130 27815 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:15:35.785655 27804 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:15:35.785966 27804 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:15:35.787135 27804 hybrid_clock.cc:648] HybridClock initialized: now 1786115735787121 us; error 29 us; skew 500 ppm
I20260807 15:15:35.788674 27804 webserver.cc:460] Webserver started at http://127.29.194.3:36701/ using document root <none> and password file <none>
I20260807 15:15:35.788902 27804 fs_manager.cc:359] Metadata directory not provided
I20260807 15:15:35.788952 27804 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:15:35.789081 27804 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:15:35.790257 27804 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115480851507-30472-0/minicluster-data/ts-2/data/instance:
uuid: "2180e69882874aa1b4a4b606a68547f3"
format_stamp: "Formatted at 2026-08-07 15:15:35 on dist-test-slave-42z9"
server_key: "2f990d4e1fbfdd9da1db2cdd0027d088"
server_key_iv: "a33d3487bb1f754f6c68b0b10d682ba0"
server_key_version: "encryptionkey@0"
I20260807 15:15:35.790620 27804 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115480851507-30472-0/minicluster-data/ts-2/wal/instance:
uuid: "2180e69882874aa1b4a4b606a68547f3"
format_stamp: "Formatted at 2026-08-07 15:15:35 on dist-test-slave-42z9"
server_key: "2f990d4e1fbfdd9da1db2cdd0027d088"
server_key_iv: "a33d3487bb1f754f6c68b0b10d682ba0"
server_key_version: "encryptionkey@0"
I20260807 15:15:35.791945 27804 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260807 15:15:35.792709 27824 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:15:35.792871 27804 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 15:15:35.792956 27804 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115480851507-30472-0/minicluster-data/ts-2/data,/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115480851507-30472-0/minicluster-data/ts-2/wal
uuid: "2180e69882874aa1b4a4b606a68547f3"
format_stamp: "Formatted at 2026-08-07 15:15:35 on dist-test-slave-42z9"
server_key: "2f990d4e1fbfdd9da1db2cdd0027d088"
server_key_iv: "a33d3487bb1f754f6c68b0b10d682ba0"
server_key_version: "encryptionkey@0"
I20260807 15:15:35.793030 27804 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115480851507-30472-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115480851507-30472-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115480851507-30472-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:15:35.809599 27804 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:15:35.809935 27804 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:15:35.810076 27804 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:15:35.810319 27804 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:15:35.810693 27804 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:15:35.810727 27804 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:15:35.810758 27804 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:15:35.810776 27804 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:15:35.816604 27804 rpc_server.cc:307] RPC server started. Bound to: 127.29.194.3:38221
I20260807 15:15:35.816660 27988 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.194.3:38221 every 8 connection(s)
I20260807 15:15:35.817003 27804 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115480851507-30472-0/minicluster-data/ts-2/data/info.pb
I20260807 15:15:35.819886 30472 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7fZSIL/build/release/bin/kudu as pid 27804
I20260807 15:15:35.820024 30472 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115480851507-30472-0/minicluster-data/ts-2/wal/instance
I20260807 15:15:35.820228 30472 external_mini_cluster.cc:1428] Setting key 05b327643595f7b78bf106f72a0dfaa2
I20260807 15:15:35.823592 27393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:35.817786 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.194.3:35505 (local address 127.29.194.62:36371)
0807 15:15:35.817855 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:35.817858 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:35.818103 (+ 245us) server_negotiation.cc:407] Connection header received
0807 15:15:35.818642 (+ 539us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:35.818644 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:35.818655 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:35.818715 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:15:35.819506 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:35.820240 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:35.822071 (+ 1831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:35.822288 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:35.822356 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:35.823366 (+ 1010us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:15:35.823368 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:15:35.823368 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:15:35.823372 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:15:35.823385 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:15:35.823389 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:15:35.823431 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0807 15:15:35.823494 (+ 63us) server_negotiation.cc:299] Negotiation successful
0807 15:15:35.823527 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260807 15:15:35.823844 27991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:35.817753 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:36371 (local address 127.29.194.3:35505)
0807 15:15:35.817984 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:35.818021 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0807 15:15:35.818549 (+ 528us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:15:35.818833 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:15:35.818837 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:15:35.818846 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:15:35.819393 (+ 547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:35.819396 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:35.820540 (+ 1144us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:35.820543 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:35.821969 (+ 1426us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:35.821974 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:35.822426 (+ 452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:35.822428 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:35.822615 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:35.823219 (+ 604us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 15:15:35.823219 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:15:35.823252 (+ 33us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 15:15:35.823282 (+ 30us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:15:35.823438 (+ 156us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:15:35.823441 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:15:35.823444 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 15:15:35.823492 (+ 48us) client_negotiation.cc:240] Negotiation successful
0807 15:15:35.823600 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":86,"threads_started":1}
I20260807 15:15:35.824286 27989 heartbeater.cc:347] Connected to a master server at 127.29.194.62:36371
I20260807 15:15:35.824378 27989 heartbeater.cc:464] Registering TS with master...
I20260807 15:15:35.824544 27989 heartbeater.cc:511] Master 127.29.194.62:36371 requested a full tablet report, sending...
I20260807 15:15:35.824867 27327 ts_manager.cc:194] Registered new tserver with Master: 2180e69882874aa1b4a4b606a68547f3 (127.29.194.3:38221)
I20260807 15:15:35.825242 27327 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.29.194.3:35505
I20260807 15:15:35.833566 30472 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 15:15:35.838557 27393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:35.835406 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36486 (local address 127.29.194.62:36371)
0807 15:15:35.835478 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:35.835481 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:35.835754 (+ 273us) server_negotiation.cc:407] Connection header received
0807 15:15:35.835845 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:35.835847 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:35.835875 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:35.835918 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:15:35.836092 (+ 174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:35.836595 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:35.837316 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:35.837595 (+ 279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:35.837654 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:35.838285 (+ 631us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:15:35.838287 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:15:35.838287 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:15:35.838292 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:15:35.838307 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:15:35.838311 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:15:35.838355 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0807 15:15:35.838418 (+ 63us) server_negotiation.cc:299] Negotiation successful
0807 15:15:35.838450 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260807 15:15:35.840193 27327 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:36486:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 15:15:35.841817 27327 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 15:15:35.849568 28008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:35.844215 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.1:44623 (local address 127.0.0.1:45920)
0807 15:15:35.844634 (+ 419us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:35.844644 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 15:15:35.844693 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:15:35.845021 (+ 328us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:15:35.845026 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:15:35.845029 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:15:35.845137 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:35.845139 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:35.846564 (+ 1425us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:35.846565 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:35.847864 (+ 1299us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:35.847866 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:35.849208 (+ 1342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:35.849209 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:35.849304 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:35.849317 (+ 13us) client_negotiation.cc:769] Sending connection context
0807 15:15:35.849399 (+ 82us) client_negotiation.cc:240] Negotiation successful
0807 15:15:35.849480 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":402,"mutex_wait_us":35,"thread_start_us":79,"threads_started":1}
I20260807 15:15:35.849689 28009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:35.844441 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45920 (local address 127.29.194.1:44623)
0807 15:15:35.844809 (+ 368us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:35.844813 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:35.844823 (+ 10us) server_negotiation.cc:407] Connection header received
0807 15:15:35.844863 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:35.844864 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:35.844880 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:35.844953 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:15:35.845221 (+ 268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:35.846469 (+ 1248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:35.847947 (+ 1478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:35.849004 (+ 1057us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:35.849066 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:35.849174 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0807 15:15:35.849470 (+ 296us) server_negotiation.cc:299] Negotiation successful
0807 15:15:35.849517 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":304,"thread_start_us":104,"threads_started":1}
I20260807 15:15:35.850296 28001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:35.843993 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.2:34341 (local address 127.0.0.1:37606)
0807 15:15:35.844298 (+ 305us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:35.844337 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0807 15:15:35.844410 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:15:35.845027 (+ 617us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:15:35.845028 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:15:35.845030 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:15:35.845137 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:35.845140 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:35.846866 (+ 1726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:35.846867 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:35.847929 (+ 1062us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:35.847932 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:35.849926 (+ 1994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:35.849927 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:35.850034 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:35.850039 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 15:15:35.850128 (+ 89us) client_negotiation.cc:240] Negotiation successful
0807 15:15:35.850206 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":278,"mutex_wait_us":34,"thread_start_us":75,"threads_started":1}
I20260807 15:15:35.850536 28006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:35.844139 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37606 (local address 127.29.194.2:34341)
0807 15:15:35.844815 (+ 676us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:35.844820 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:35.844830 (+ 10us) server_negotiation.cc:407] Connection header received
0807 15:15:35.844869 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:35.844871 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:35.844887 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:35.844960 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:15:35.845223 (+ 263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:35.846782 (+ 1559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:35.849081 (+ 2299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:35.849813 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:35.849887 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:35.849971 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0807 15:15:35.850401 (+ 430us) server_negotiation.cc:299] Negotiation successful
0807 15:15:35.850452 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":635,"thread_start_us":120,"threads_started":1}
I20260807 15:15:35.851015 28002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:35.844007 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.3:38221 (local address 127.0.0.1:56148)
0807 15:15:35.844364 (+ 357us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:35.844375 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 15:15:35.845192 (+ 817us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:15:35.845524 (+ 332us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:15:35.845525 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:15:35.845528 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:15:35.845576 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:35.845578 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:35.846809 (+ 1231us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:35.846810 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:35.848960 (+ 2150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:35.848964 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:35.850824 (+ 1860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:35.850825 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:35.850899 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:35.850903 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 15:15:35.850938 (+ 35us) client_negotiation.cc:240] Negotiation successful
0807 15:15:35.850968 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":339,"mutex_wait_us":2,"thread_start_us":113,"threads_started":1}
I20260807 15:15:35.851325 27701 tablet_service.cc:1467] Processing CreateTablet for tablet 59d2c822ceba49ed95d18ee29af4e1d5 (DEFAULT_TABLE table=test-table [id=2f28bd679e4e42438714cdc96dfead1a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:15:35.851760 27701 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 59d2c822ceba49ed95d18ee29af4e1d5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:15:35.854455 27504 tablet_service.cc:1467] Processing CreateTablet for tablet 59d2c822ceba49ed95d18ee29af4e1d5 (DEFAULT_TABLE table=test-table [id=2f28bd679e4e42438714cdc96dfead1a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:15:35.854799 27504 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 59d2c822ceba49ed95d18ee29af4e1d5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:15:35.856070 28012 tablet_bootstrap.cc:492] T 59d2c822ceba49ed95d18ee29af4e1d5 P 5e5d467cacee4c26b6ae9ca20f8be7ed: Bootstrap starting.
I20260807 15:15:35.856658 28005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:35.844119 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56148 (local address 127.29.194.3:38221)
0807 15:15:35.844581 (+ 462us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:35.844586 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:35.844597 (+ 11us) server_negotiation.cc:407] Connection header received
0807 15:15:35.845353 (+ 756us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:35.845355 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:35.845371 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:35.845434 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:15:35.845657 (+ 223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:35.846707 (+ 1050us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:35.850014 (+ 3307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:35.850733 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:35.856347 (+ 5614us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:35.856482 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0807 15:15:35.856516 (+ 34us) server_negotiation.cc:299] Negotiation successful
0807 15:15:35.856571 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":344,"thread_start_us":85,"threads_started":1}
I20260807 15:15:35.856995 28012 tablet_bootstrap.cc:654] T 59d2c822ceba49ed95d18ee29af4e1d5 P 5e5d467cacee4c26b6ae9ca20f8be7ed: Neither blocks nor log segments found. Creating new log.
I20260807 15:15:35.857384 28012 log.cc:824] T 59d2c822ceba49ed95d18ee29af4e1d5 P 5e5d467cacee4c26b6ae9ca20f8be7ed: Log is configured to *not* fsync() on all Append() calls
I20260807 15:15:35.857350 27896 tablet_service.cc:1467] Processing CreateTablet for tablet 59d2c822ceba49ed95d18ee29af4e1d5 (DEFAULT_TABLE table=test-table [id=2f28bd679e4e42438714cdc96dfead1a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:15:35.857739 27896 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 59d2c822ceba49ed95d18ee29af4e1d5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:15:35.857987 28015 tablet_bootstrap.cc:492] T 59d2c822ceba49ed95d18ee29af4e1d5 P 2c749b22952e480e8b786b3287e678e6: Bootstrap starting.
I20260807 15:15:35.858651 28015 tablet_bootstrap.cc:654] T 59d2c822ceba49ed95d18ee29af4e1d5 P 2c749b22952e480e8b786b3287e678e6: Neither blocks nor log segments found. Creating new log.
I20260807 15:15:35.858961 28015 log.cc:824] T 59d2c822ceba49ed95d18ee29af4e1d5 P 2c749b22952e480e8b786b3287e678e6: Log is configured to *not* fsync() on all Append() calls
I20260807 15:15:35.859826 28015 tablet_bootstrap.cc:492] T 59d2c822ceba49ed95d18ee29af4e1d5 P 2c749b22952e480e8b786b3287e678e6: No bootstrap required, opened a new log
I20260807 15:15:35.859859 28012 tablet_bootstrap.cc:492] T 59d2c822ceba49ed95d18ee29af4e1d5 P 5e5d467cacee4c26b6ae9ca20f8be7ed: No bootstrap required, opened a new log
I20260807 15:15:35.859901 28015 ts_tablet_manager.cc:1397] T 59d2c822ceba49ed95d18ee29af4e1d5 P 2c749b22952e480e8b786b3287e678e6: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 15:15:35.859920 28012 ts_tablet_manager.cc:1397] T 59d2c822ceba49ed95d18ee29af4e1d5 P 5e5d467cacee4c26b6ae9ca20f8be7ed: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.001s
I20260807 15:15:35.861106 28018 tablet_bootstrap.cc:492] T 59d2c822ceba49ed95d18ee29af4e1d5 P 2180e69882874aa1b4a4b606a68547f3: Bootstrap starting.
I20260807 15:15:35.861876 28018 tablet_bootstrap.cc:654] T 59d2c822ceba49ed95d18ee29af4e1d5 P 2180e69882874aa1b4a4b606a68547f3: Neither blocks nor log segments found. Creating new log.
I20260807 15:15:35.862207 28018 log.cc:824] T 59d2c822ceba49ed95d18ee29af4e1d5 P 2180e69882874aa1b4a4b606a68547f3: Log is configured to *not* fsync() on all Append() calls
I20260807 15:15:35.862219 28012 raft_consensus.cc:348] T 59d2c822ceba49ed95d18ee29af4e1d5 P 5e5d467cacee4c26b6ae9ca20f8be7ed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e5d467cacee4c26b6ae9ca20f8be7ed" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 34341 } } peers { permanent_uuid: "2180e69882874aa1b4a4b606a68547f3" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 38221 } } peers { permanent_uuid: "2c749b22952e480e8b786b3287e678e6" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 44623 } }
I20260807 15:15:35.862385 28012 raft_consensus.cc:374] T 59d2c822ceba49ed95d18ee29af4e1d5 P 5e5d467cacee4c26b6ae9ca20f8be7ed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:15:35.862428 28012 raft_consensus.cc:729] T 59d2c822ceba49ed95d18ee29af4e1d5 P 5e5d467cacee4c26b6ae9ca20f8be7ed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e5d467cacee4c26b6ae9ca20f8be7ed, State: Initialized, Role: FOLLOWER
I20260807 15:15:35.862566 28012 consensus_queue.cc:260] T 59d2c822ceba49ed95d18ee29af4e1d5 P 5e5d467cacee4c26b6ae9ca20f8be7ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e5d467cacee4c26b6ae9ca20f8be7ed" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 34341 } } peers { permanent_uuid: "2180e69882874aa1b4a4b606a68547f3" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 38221 } } peers { permanent_uuid: "2c749b22952e480e8b786b3287e678e6" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 44623 } }
I20260807 15:15:35.862572 28015 raft_consensus.cc:348] T 59d2c822ceba49ed95d18ee29af4e1d5 P 2c749b22952e480e8b786b3287e678e6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e5d467cacee4c26b6ae9ca20f8be7ed" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 34341 } } peers { permanent_uuid: "2180e69882874aa1b4a4b606a68547f3" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 38221 } } peers { permanent_uuid: "2c749b22952e480e8b786b3287e678e6" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 44623 } }
I20260807 15:15:35.862731 28015 raft_consensus.cc:374] T 59d2c822ceba49ed95d18ee29af4e1d5 P 2c749b22952e480e8b786b3287e678e6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:15:35.862779 28015 raft_consensus.cc:729] T 59d2c822ceba49ed95d18ee29af4e1d5 P 2c749b22952e480e8b786b3287e678e6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c749b22952e480e8b786b3287e678e6, State: Initialized, Role: FOLLOWER
I20260807 15:15:35.862802 28012 ts_tablet_manager.cc:1428] T 59d2c822ceba49ed95d18ee29af4e1d5 P 5e5d467cacee4c26b6ae9ca20f8be7ed: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260807 15:15:35.862941 28018 tablet_bootstrap.cc:492] T 59d2c822ceba49ed95d18ee29af4e1d5 P 2180e69882874aa1b4a4b606a68547f3: No bootstrap required, opened a new log
I20260807 15:15:35.862941 27799 heartbeater.cc:503] Master 127.29.194.62:36371 was elected leader, sending a full tablet report...
I20260807 15:15:35.862928 28015 consensus_queue.cc:260] T 59d2c822ceba49ed95d18ee29af4e1d5 P 2c749b22952e480e8b786b3287e678e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e5d467cacee4c26b6ae9ca20f8be7ed" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 34341 } } peers { permanent_uuid: "2180e69882874aa1b4a4b606a68547f3" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 38221 } } peers { permanent_uuid: "2c749b22952e480e8b786b3287e678e6" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 44623 } }
I20260807 15:15:35.863003 28018 ts_tablet_manager.cc:1397] T 59d2c822ceba49ed95d18ee29af4e1d5 P 2180e69882874aa1b4a4b606a68547f3: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260807 15:15:35.863185 28015 ts_tablet_manager.cc:1428] T 59d2c822ceba49ed95d18ee29af4e1d5 P 2c749b22952e480e8b786b3287e678e6: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260807 15:15:35.863230 27608 heartbeater.cc:503] Master 127.29.194.62:36371 was elected leader, sending a full tablet report...
I20260807 15:15:35.865088 28018 raft_consensus.cc:348] T 59d2c822ceba49ed95d18ee29af4e1d5 P 2180e69882874aa1b4a4b606a68547f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e5d467cacee4c26b6ae9ca20f8be7ed" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 34341 } } peers { permanent_uuid: "2180e69882874aa1b4a4b606a68547f3" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 38221 } } peers { permanent_uuid: "2c749b22952e480e8b786b3287e678e6" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 44623 } }
I20260807 15:15:35.865214 28018 raft_consensus.cc:374] T 59d2c822ceba49ed95d18ee29af4e1d5 P 2180e69882874aa1b4a4b606a68547f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:15:35.865234 28018 raft_consensus.cc:729] T 59d2c822ceba49ed95d18ee29af4e1d5 P 2180e69882874aa1b4a4b606a68547f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2180e69882874aa1b4a4b606a68547f3, State: Initialized, Role: FOLLOWER
I20260807 15:15:35.865348 28018 consensus_queue.cc:260] T 59d2c822ceba49ed95d18ee29af4e1d5 P 2180e69882874aa1b4a4b606a68547f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e5d467cacee4c26b6ae9ca20f8be7ed" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 34341 } } peers { permanent_uuid: "2180e69882874aa1b4a4b606a68547f3" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 38221 } } peers { permanent_uuid: "2c749b22952e480e8b786b3287e678e6" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 44623 } }
I20260807 15:15:35.865602 28018 ts_tablet_manager.cc:1428] T 59d2c822ceba49ed95d18ee29af4e1d5 P 2180e69882874aa1b4a4b606a68547f3: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260807 15:15:35.865648 27989 heartbeater.cc:503] Master 127.29.194.62:36371 was elected leader, sending a full tablet report...
W20260807 15:15:35.934037 27802 tablet.cc:2369] T 59d2c822ceba49ed95d18ee29af4e1d5 P 5e5d467cacee4c26b6ae9ca20f8be7ed: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 15:15:36.058166 27610 tablet.cc:2369] T 59d2c822ceba49ed95d18ee29af4e1d5 P 2c749b22952e480e8b786b3287e678e6: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 15:15:36.067593 27990 tablet.cc:2369] T 59d2c822ceba49ed95d18ee29af4e1d5 P 2180e69882874aa1b4a4b606a68547f3: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 15:15:36.215727 28021 raft_consensus.cc:482] T 59d2c822ceba49ed95d18ee29af4e1d5 P 5e5d467cacee4c26b6ae9ca20f8be7ed [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 15:15:36.215883 28021 raft_consensus.cc:504] T 59d2c822ceba49ed95d18ee29af4e1d5 P 5e5d467cacee4c26b6ae9ca20f8be7ed [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e5d467cacee4c26b6ae9ca20f8be7ed" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 34341 } } peers { permanent_uuid: "2180e69882874aa1b4a4b606a68547f3" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 38221 } } peers { permanent_uuid: "2c749b22952e480e8b786b3287e678e6" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 44623 } }
I20260807 15:15:36.216269 28021 leader_election.cc:288] T 59d2c822ceba49ed95d18ee29af4e1d5 P 5e5d467cacee4c26b6ae9ca20f8be7ed [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2180e69882874aa1b4a4b606a68547f3 (127.29.194.3:38221), 2c749b22952e480e8b786b3287e678e6 (127.29.194.1:44623)
I20260807 15:15:36.219251 28009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:36.216747 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.194.2:37697 (local address 127.29.194.1:44623)
0807 15:15:36.216802 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:36.216804 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:36.216814 (+ 10us) server_negotiation.cc:407] Connection header received
0807 15:15:36.216858 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:36.216859 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:36.216869 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:36.216915 (+ 46us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:15:36.217075 (+ 160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:36.217592 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:36.218559 (+ 967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:36.218912 (+ 353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:36.218975 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:36.218983 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0807 15:15:36.219137 (+ 154us) server_negotiation.cc:299] Negotiation successful
0807 15:15:36.219176 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260807 15:15:36.219251 28005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:36.216595 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.194.2:45087 (local address 127.29.194.3:38221)
0807 15:15:36.216660 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:36.216662 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:36.216732 (+ 70us) server_negotiation.cc:407] Connection header received
0807 15:15:36.216828 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:36.216829 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:36.216838 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:36.216888 (+ 50us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:15:36.217041 (+ 153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:36.217542 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:36.218551 (+ 1009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:36.218913 (+ 362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:36.218975 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:36.218984 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0807 15:15:36.219134 (+ 150us) server_negotiation.cc:299] Negotiation successful
0807 15:15:36.219174 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260807 15:15:36.219281 28026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:36.216660 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.1:44623 (local address 127.29.194.2:37697)
0807 15:15:36.216740 (+ 80us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:36.216750 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 15:15:36.216807 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:15:36.216926 (+ 119us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:15:36.216927 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:15:36.216928 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:15:36.217001 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:36.217004 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:36.217671 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:36.217672 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:36.218462 (+ 790us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:36.218464 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:36.218987 (+ 523us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:36.218988 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:36.219074 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:36.219078 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 15:15:36.219123 (+ 45us) client_negotiation.cc:240] Negotiation successful
0807 15:15:36.219167 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":65,"mutex_wait_us":1,"thread_start_us":39,"threads_started":1}
I20260807 15:15:36.219285 28025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:36.216471 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.3:38221 (local address 127.29.194.2:45087)
0807 15:15:36.216636 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:36.216652 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 15:15:36.216764 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:15:36.216902 (+ 138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:15:36.216903 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:15:36.216905 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:15:36.216979 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:36.216981 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:36.217622 (+ 641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:36.217623 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:36.218450 (+ 827us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:15:36.218452 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:15:36.219000 (+ 548us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:15:36.219002 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:15:36.219071 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:36.219074 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 15:15:36.219123 (+ 49us) client_negotiation.cc:240] Negotiation successful
0807 15:15:36.219168 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"mutex_wait_us":30,"thread_start_us":76,"threads_started":1}
I20260807 15:15:36.219669 27534 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59d2c822ceba49ed95d18ee29af4e1d5" candidate_uuid: "5e5d467cacee4c26b6ae9ca20f8be7ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c749b22952e480e8b786b3287e678e6" is_pre_election: true
I20260807 15:15:36.219872 27534 raft_consensus.cc:2454] T 59d2c822ceba49ed95d18ee29af4e1d5 P 2c749b22952e480e8b786b3287e678e6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5d467cacee4c26b6ae9ca20f8be7ed in term 0.
I20260807 15:15:36.219835 27926 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59d2c822ceba49ed95d18ee29af4e1d5" candidate_uuid: "5e5d467cacee4c26b6ae9ca20f8be7ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2180e69882874aa1b4a4b606a68547f3" is_pre_election: true
I20260807 15:15:36.219964 27926 raft_consensus.cc:2454] T 59d2c822ceba49ed95d18ee29af4e1d5 P 2180e69882874aa1b4a4b606a68547f3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5d467cacee4c26b6ae9ca20f8be7ed in term 0.
I20260807 15:15:36.220155 27638 leader_election.cc:302] T 59d2c822ceba49ed95d18ee29af4e1d5 P 5e5d467cacee4c26b6ae9ca20f8be7ed [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c749b22952e480e8b786b3287e678e6, 5e5d467cacee4c26b6ae9ca20f8be7ed; no voters:
I20260807 15:15:36.220338 28021 raft_consensus.cc:2781] T 59d2c822ceba49ed95d18ee29af4e1d5 P 5e5d467cacee4c26b6ae9ca20f8be7ed [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 15:15:36.220425 28021 raft_consensus.cc:482] T 59d2c822ceba49ed95d18ee29af4e1d5 P 5e5d467cacee4c26b6ae9ca20f8be7ed [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 15:15:36.220446 28021 raft_consensus.cc:3037] T 59d2c822ceba49ed95d18ee29af4e1d5 P 5e5d467cacee4c26b6ae9ca20f8be7ed [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:15:36.221235 28021 raft_consensus.cc:504] T 59d2c822ceba49ed95d18ee29af4e1d5 P 5e5d467cacee4c26b6ae9ca20f8be7ed [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e5d467cacee4c26b6ae9ca20f8be7ed" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 34341 } } peers { permanent_uuid: "2180e69882874aa1b4a4b606a68547f3" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 38221 } } peers { permanent_uuid: "2c749b22952e480e8b786b3287e678e6" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 44623 } }
I20260807 15:15:36.221359 28021 leader_election.cc:288] T 59d2c822ceba49ed95d18ee29af4e1d5 P 5e5d467cacee4c26b6ae9ca20f8be7ed [CANDIDATE]: Term 1 election: Requested vote from peers 2180e69882874aa1b4a4b606a68547f3 (127.29.194.3:38221), 2c749b22952e480e8b786b3287e678e6 (127.29.194.1:44623)
I20260807 15:15:36.221843 27534 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59d2c822ceba49ed95d18ee29af4e1d5" candidate_uuid: "5e5d467cacee4c26b6ae9ca20f8be7ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c749b22952e480e8b786b3287e678e6"
I20260807 15:15:36.221808 27926 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59d2c822ceba49ed95d18ee29af4e1d5" candidate_uuid: "5e5d467cacee4c26b6ae9ca20f8be7ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2180e69882874aa1b4a4b606a68547f3"
I20260807 15:15:36.221937 27534 raft_consensus.cc:3037] T 59d2c822ceba49ed95d18ee29af4e1d5 P 2c749b22952e480e8b786b3287e678e6 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:15:36.221989 27926 raft_consensus.cc:3037] T 59d2c822ceba49ed95d18ee29af4e1d5 P 2180e69882874aa1b4a4b606a68547f3 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:15:36.222733 27534 raft_consensus.cc:2454] T 59d2c822ceba49ed95d18ee29af4e1d5 P 2c749b22952e480e8b786b3287e678e6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5d467cacee4c26b6ae9ca20f8be7ed in term 1.
I20260807 15:15:36.222939 27638 leader_election.cc:302] T 59d2c822ceba49ed95d18ee29af4e1d5 P 5e5d467cacee4c26b6ae9ca20f8be7ed [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c749b22952e480e8b786b3287e678e6, 5e5d467cacee4c26b6ae9ca20f8be7ed; no voters:
I20260807 15:15:36.222934 27926 raft_consensus.cc:2454] T 59d2c822ceba49ed95d18ee29af4e1d5 P 2180e69882874aa1b4a4b606a68547f3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5d467cacee4c26b6ae9ca20f8be7ed in term 1.
I20260807 15:15:36.223132 28021 raft_consensus.cc:2781] T 59d2c822ceba49ed95d18ee29af4e1d5 P 5e5d467cacee4c26b6ae9ca20f8be7ed [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:15:36.223296 28021 raft_consensus.cc:686] T 59d2c822ceba49ed95d18ee29af4e1d5 P 5e5d467cacee4c26b6ae9ca20f8be7ed [term 1 LEADER]: Becoming Leader. State: Replica: 5e5d467cacee4c26b6ae9ca20f8be7ed, State: Running, Role: LEADER
I20260807 15:15:36.223384 28021 consensus_queue.cc:237] T 59d2c822ceba49ed95d18ee29af4e1d5 P 5e5d467cacee4c26b6ae9ca20f8be7ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e5d467cacee4c26b6ae9ca20f8be7ed" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 34341 } } peers { permanent_uuid: "2180e69882874aa1b4a4b606a68547f3" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 38221 } } peers { permanent_uuid: "2c749b22952e480e8b786b3287e678e6" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 44623 } }
I20260807 15:15:36.224210 27326 catalog_manager.cc:5495] T 59d2c822ceba49ed95d18ee29af4e1d5 P 5e5d467cacee4c26b6ae9ca20f8be7ed reported cstate change: term changed from 0 to 1, leader changed from <none> to 5e5d467cacee4c26b6ae9ca20f8be7ed (127.29.194.2). New cstate: current_term: 1 leader_uuid: "5e5d467cacee4c26b6ae9ca20f8be7ed" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e5d467cacee4c26b6ae9ca20f8be7ed" member_type: VOTER last_known_addr { host: "127.29.194.2" port: 34341 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2180e69882874aa1b4a4b606a68547f3" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 38221 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c749b22952e480e8b786b3287e678e6" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 44623 } health_report { overall_health: UNKNOWN } } }
I20260807 15:15:36.292358 28006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:36.290167 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37608 (local address 127.29.194.2:34341)
0807 15:15:36.290236 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:36.290237 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:36.290246 (+ 9us) server_negotiation.cc:407] Connection header received
0807 15:15:36.290288 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:36.290289 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:36.290308 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:36.290373 (+ 65us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 15:15:36.290468 (+ 95us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:36.290937 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:36.291713 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:36.291906 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:36.291964 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:36.292080 (+ 116us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 15:15:36.292186 (+ 106us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 15:15:36.292243 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0807 15:15:36.292256 (+ 13us) server_negotiation.cc:299] Negotiation successful
0807 15:15:36.292273 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260807 15:15:36.294476 27926 raft_consensus.cc:1261] T 59d2c822ceba49ed95d18ee29af4e1d5 P 2180e69882874aa1b4a4b606a68547f3 [term 1 FOLLOWER]: Refusing update from remote peer 5e5d467cacee4c26b6ae9ca20f8be7ed: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:15:36.294534 27534 raft_consensus.cc:1261] T 59d2c822ceba49ed95d18ee29af4e1d5 P 2c749b22952e480e8b786b3287e678e6 [term 1 FOLLOWER]: Refusing update from remote peer 5e5d467cacee4c26b6ae9ca20f8be7ed: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:15:36.294867 28027 consensus_queue.cc:1035] T 59d2c822ceba49ed95d18ee29af4e1d5 P 5e5d467cacee4c26b6ae9ca20f8be7ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "2180e69882874aa1b4a4b606a68547f3" member_type: VOTER last_known_addr { host: "127.29.194.3" port: 38221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:15:36.295002 28021 consensus_queue.cc:1035] T 59d2c822ceba49ed95d18ee29af4e1d5 P 5e5d467cacee4c26b6ae9ca20f8be7ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c749b22952e480e8b786b3287e678e6" member_type: VOTER last_known_addr { host: "127.29.194.1" port: 44623 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:15:36.297461 28032 mvcc.cc:205] Tried to move back new op lower bound from 7315930055858737152 to 7315930055571615744. Current Snapshot: MvccSnapshot[applied={T|T < 7315930055858737152}]
I20260807 15:15:36.297847 28037 mvcc.cc:205] Tried to move back new op lower bound from 7315930055858737152 to 7315930055571615744. Current Snapshot: MvccSnapshot[applied={T|T < 7315930055858737152}]
I20260807 15:15:36.298000 28039 mvcc.cc:205] Tried to move back new op lower bound from 7315930055858737152 to 7315930055571615744. Current Snapshot: MvccSnapshot[applied={T|T < 7315930055858737152}]
I20260807 15:15:36.299698 27326 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:36486:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 15:15:36.299793 27326 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:36486:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 15:15:36.300547 27326 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e766bca9bc9a4dd695fb50d9e947ecde: Sending DeleteTablet for 3 replicas of tablet 59d2c822ceba49ed95d18ee29af4e1d5
I20260807 15:15:36.300925 27701 tablet_service.cc:1514] Processing DeleteTablet for tablet 59d2c822ceba49ed95d18ee29af4e1d5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:15:36 UTC) from {username='slave'} at 127.0.0.1:37606
I20260807 15:15:36.300933 27504 tablet_service.cc:1514] Processing DeleteTablet for tablet 59d2c822ceba49ed95d18ee29af4e1d5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:15:36 UTC) from {username='slave'} at 127.0.0.1:45920
I20260807 15:15:36.301160 28051 tablet_replica.cc:330] T 59d2c822ceba49ed95d18ee29af4e1d5 P 2c749b22952e480e8b786b3287e678e6: stopping tablet replica
I20260807 15:15:36.301192 27896 tablet_service.cc:1514] Processing DeleteTablet for tablet 59d2c822ceba49ed95d18ee29af4e1d5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:15:36 UTC) from {username='slave'} at 127.0.0.1:56148
I20260807 15:15:36.301278 28051 raft_consensus.cc:2229] T 59d2c822ceba49ed95d18ee29af4e1d5 P 2c749b22952e480e8b786b3287e678e6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 15:15:36.301414 28051 raft_consensus.cc:2258] T 59d2c822ceba49ed95d18ee29af4e1d5 P 2c749b22952e480e8b786b3287e678e6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:15:36.301723 30472 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7fZSIL/build/release/bin/kudu with pid 27421
I20260807 15:15:36.301904 28051 ts_tablet_manager.cc:1905] T 59d2c822ceba49ed95d18ee29af4e1d5 P 2c749b22952e480e8b786b3287e678e6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:15:36.307595 28053 tablet_replica.cc:330] T 59d2c822ceba49ed95d18ee29af4e1d5 P 5e5d467cacee4c26b6ae9ca20f8be7ed: stopping tablet replica
I20260807 15:15:36.307749 28053 raft_consensus.cc:2229] T 59d2c822ceba49ed95d18ee29af4e1d5 P 5e5d467cacee4c26b6ae9ca20f8be7ed [term 1 LEADER]: Raft consensus shutting down.
I20260807 15:15:36.307905 28053 raft_consensus.cc:2258] T 59d2c822ceba49ed95d18ee29af4e1d5 P 5e5d467cacee4c26b6ae9ca20f8be7ed [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:15:36.308190 28054 tablet_replica.cc:330] T 59d2c822ceba49ed95d18ee29af4e1d5 P 2180e69882874aa1b4a4b606a68547f3: stopping tablet replica
I20260807 15:15:36.308316 28054 raft_consensus.cc:2229] T 59d2c822ceba49ed95d18ee29af4e1d5 P 2180e69882874aa1b4a4b606a68547f3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 15:15:36.308431 28054 raft_consensus.cc:2258] T 59d2c822ceba49ed95d18ee29af4e1d5 P 2180e69882874aa1b4a4b606a68547f3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:15:36.308619 28053 ts_tablet_manager.cc:1905] T 59d2c822ceba49ed95d18ee29af4e1d5 P 5e5d467cacee4c26b6ae9ca20f8be7ed: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:15:36.308899 28054 ts_tablet_manager.cc:1905] T 59d2c822ceba49ed95d18ee29af4e1d5 P 2180e69882874aa1b4a4b606a68547f3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:15:36.310416 28053 ts_tablet_manager.cc:1918] T 59d2c822ceba49ed95d18ee29af4e1d5 P 5e5d467cacee4c26b6ae9ca20f8be7ed: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 15:15:36.310451 28054 ts_tablet_manager.cc:1918] T 59d2c822ceba49ed95d18ee29af4e1d5 P 2180e69882874aa1b4a4b606a68547f3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 15:15:36.310531 28054 log.cc:1196] T 59d2c822ceba49ed95d18ee29af4e1d5 P 2180e69882874aa1b4a4b606a68547f3: Deleting WAL directory at /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115480851507-30472-0/minicluster-data/ts-2/wal/wals/59d2c822ceba49ed95d18ee29af4e1d5
I20260807 15:15:36.310639 28053 log.cc:1196] T 59d2c822ceba49ed95d18ee29af4e1d5 P 5e5d467cacee4c26b6ae9ca20f8be7ed: Deleting WAL directory at /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115480851507-30472-0/minicluster-data/ts-1/wal/wals/59d2c822ceba49ed95d18ee29af4e1d5
I20260807 15:15:36.310786 28054 ts_tablet_manager.cc:1939] T 59d2c822ceba49ed95d18ee29af4e1d5 P 2180e69882874aa1b4a4b606a68547f3: Deleting consensus metadata
I20260807 15:15:36.311013 28053 ts_tablet_manager.cc:1939] T 59d2c822ceba49ed95d18ee29af4e1d5 P 5e5d467cacee4c26b6ae9ca20f8be7ed: Deleting consensus metadata
I20260807 15:15:36.311266 27311 catalog_manager.cc:4841] TS 2180e69882874aa1b4a4b606a68547f3 (127.29.194.3:38221): tablet 59d2c822ceba49ed95d18ee29af4e1d5 (table test-table [id=2f28bd679e4e42438714cdc96dfead1a]) successfully deleted
I20260807 15:15:36.311437 27307 catalog_manager.cc:4841] TS 5e5d467cacee4c26b6ae9ca20f8be7ed (127.29.194.2:34341): tablet 59d2c822ceba49ed95d18ee29af4e1d5 (table test-table [id=2f28bd679e4e42438714cdc96dfead1a]) successfully deleted
W20260807 15:15:36.312729 27311 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.29.194.1:44623 (error 108)
I20260807 15:15:36.312976 30472 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7fZSIL/build/release/bin/kudu with pid 27612
I20260807 15:15:36.315387 28002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:36.313157 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.1:44623 (local address 127.0.0.1:45930)
0807 15:15:36.315289 (+ 2132us) negotiation.cc:107] Waiting for socket to connect
0807 15:15:36.315350 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.194.1:44623: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2100}
W20260807 15:15:36.319293 27311 catalog_manager.cc:4584] TS 2c749b22952e480e8b786b3287e678e6 (127.29.194.1:44623): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 59d2c822ceba49ed95d18ee29af4e1d5: Network error: Client connection negotiation failed: client connection to 127.29.194.1:44623: connect: Connection refused (error 111)
I20260807 15:15:36.324537 30472 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7fZSIL/build/release/bin/kudu with pid 27804
I20260807 15:15:36.335832 30472 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7fZSIL/build/release/bin/kudu with pid 27286
2026-08-07T15:15:36Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (1113 ms)
[----------] 6 tests from EncryptionPolicyTest (9437 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786115480851507-30472-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 15:15:36 dist-test-slave-42z9 krb5kdc[28060](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 15:15:36 dist-test-slave-42z9 krb5kdc[28060](info): set up 2 sockets
Aug 07 15:15:36 dist-test-slave-42z9 krb5kdc[28060](info): commencing operation
krb5kdc: starting...
W20260807 15:15:37.481030 30472 mini_kdc.cc:121] Time spent starting KDC: real 1.118s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786115480851507-30472-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786115480851507-30472-0/krb5kdc/test-user.keytab.
Aug 07 15:15:37 dist-test-slave-42z9 krb5kdc[28060](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115737, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T15:15:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T15:15:37Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786115480851507-30472-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786115480851507-30472-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 15:15:37.609403 30472 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7fZSIL/build/release/bin/kudu
/tmp/dist-test-task7fZSIL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786115480851507-30472-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786115480851507-30472-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786115480851507-30472-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786115480851507-30472-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44993
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786115480851507-30472-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36917
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44993
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786115480851507-30472-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786115480851507-30472-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786115480851507-30472-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:15:37.693766 28082 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:15:37.693993 28082 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:15:37.694018 28082 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:15:37.695252 28082 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:15:37.695305 28082 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:15:37.695319 28082 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:15:37.695330 28082 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:15:37.695341 28082 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:15:37.696905 28082 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36917
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786115480851507-30472-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786115480851507-30472-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44993
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786115480851507-30472-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44993
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786115480851507-30472-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28082
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786115480851507-30472-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:15:37.697171 28082 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:15:37.697427 28082 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:15:37.703411 28089 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:15:37.703413 28091 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:15:37.703372 28088 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:15:37.703500 28082 server_base.cc:1034] running on GCE node
I20260807 15:15:37.703855 28082 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:15:37.704175 28082 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:15:37.705370 28082 hybrid_clock.cc:648] HybridClock initialized: now 1786115737705354 us; error 30 us; skew 500 ppm
Aug 07 15:15:37 dist-test-slave-42z9 krb5kdc[28060](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115737, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:15:37.711287 28082 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 15:15:37.711748 28082 webserver.cc:460] Webserver started at http://127.0.0.1:39575/ using document root <none> and password file <none>
I20260807 15:15:37.711983 28082 fs_manager.cc:359] Metadata directory not provided
I20260807 15:15:37.712033 28082 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:15:37.712160 28082 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:15:37.713219 28082 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786115480851507-30472-0/minicluster-data/master-0/data/instance:
uuid: "3af9aa3f57c44431a8ebd226a45570f0"
format_stamp: "Formatted at 2026-08-07 15:15:37 on dist-test-slave-42z9"
server_key: "37dbcedbb3fea853e393281953741614"
server_key_iv: "b9758a63072feb4844f9ae618a3932b5"
server_key_version: "encryptionkey@0"
I20260807 15:15:37.713586 28082 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786115480851507-30472-0/minicluster-data/master-0/wal/instance:
uuid: "3af9aa3f57c44431a8ebd226a45570f0"
format_stamp: "Formatted at 2026-08-07 15:15:37 on dist-test-slave-42z9"
server_key: "37dbcedbb3fea853e393281953741614"
server_key_iv: "b9758a63072feb4844f9ae618a3932b5"
server_key_version: "encryptionkey@0"
I20260807 15:15:37.714929 28082 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260807 15:15:37.715716 28100 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:15:37.715909 28082 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 15:15:37.716010 28082 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786115480851507-30472-0/minicluster-data/master-0/data,/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786115480851507-30472-0/minicluster-data/master-0/wal
uuid: "3af9aa3f57c44431a8ebd226a45570f0"
format_stamp: "Formatted at 2026-08-07 15:15:37 on dist-test-slave-42z9"
server_key: "37dbcedbb3fea853e393281953741614"
server_key_iv: "b9758a63072feb4844f9ae618a3932b5"
server_key_version: "encryptionkey@0"
I20260807 15:15:37.716074 28082 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786115480851507-30472-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786115480851507-30472-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786115480851507-30472-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:15:37.737521 28082 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:15:37.739707 28082 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:15:37.739856 28082 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:15:37.743939 28082 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44993
I20260807 15:15:37.744007 28186 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44993 every 8 connection(s)
I20260807 15:15:37.744338 28082 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786115480851507-30472-0/minicluster-data/master-0/data/info.pb
I20260807 15:15:37.745219 28190 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:15:37.745767 30472 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7fZSIL/build/release/bin/kudu as pid 28082
I20260807 15:15:37.745888 30472 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786115480851507-30472-0/minicluster-data/master-0/wal/instance
I20260807 15:15:37.746094 30472 external_mini_cluster.cc:1428] Setting key 1df1e4f199d48279c9b90233795e3c3e
I20260807 15:15:37.748214 28190 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3af9aa3f57c44431a8ebd226a45570f0: Bootstrap starting.
I20260807 15:15:37.748818 28190 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3af9aa3f57c44431a8ebd226a45570f0: Neither blocks nor log segments found. Creating new log.
I20260807 15:15:37.749101 28190 log.cc:824] T 00000000000000000000000000000000 P 3af9aa3f57c44431a8ebd226a45570f0: Log is configured to *not* fsync() on all Append() calls
I20260807 15:15:37.749893 28190 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3af9aa3f57c44431a8ebd226a45570f0: No bootstrap required, opened a new log
I20260807 15:15:37.752301 28190 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3af9aa3f57c44431a8ebd226a45570f0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3af9aa3f57c44431a8ebd226a45570f0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44993 } }
I20260807 15:15:37.752465 28190 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3af9aa3f57c44431a8ebd226a45570f0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:15:37.752490 28190 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3af9aa3f57c44431a8ebd226a45570f0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3af9aa3f57c44431a8ebd226a45570f0, State: Initialized, Role: FOLLOWER
I20260807 15:15:37.752602 28190 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3af9aa3f57c44431a8ebd226a45570f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3af9aa3f57c44431a8ebd226a45570f0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44993 } }
I20260807 15:15:37.752676 28190 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3af9aa3f57c44431a8ebd226a45570f0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:15:37.752707 28190 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3af9aa3f57c44431a8ebd226a45570f0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:15:37.752755 28190 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3af9aa3f57c44431a8ebd226a45570f0 [term 0 FOLLOWER]: Advancing to term 1
Aug 07 15:15:37 dist-test-slave-42z9 krb5kdc[28060](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115737, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260807 15:15:37.753335 28190 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3af9aa3f57c44431a8ebd226a45570f0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3af9aa3f57c44431a8ebd226a45570f0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44993 } }
I20260807 15:15:37.753460 28190 leader_election.cc:302] T 00000000000000000000000000000000 P 3af9aa3f57c44431a8ebd226a45570f0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3af9aa3f57c44431a8ebd226a45570f0; no voters:
I20260807 15:15:37.753635 28190 leader_election.cc:288] T 00000000000000000000000000000000 P 3af9aa3f57c44431a8ebd226a45570f0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 15:15:37.753672 28195 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3af9aa3f57c44431a8ebd226a45570f0 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:15:37.753841 28195 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3af9aa3f57c44431a8ebd226a45570f0 [term 1 LEADER]: Becoming Leader. State: Replica: 3af9aa3f57c44431a8ebd226a45570f0, State: Running, Role: LEADER
I20260807 15:15:37.753983 28195 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3af9aa3f57c44431a8ebd226a45570f0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3af9aa3f57c44431a8ebd226a45570f0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44993 } }
I20260807 15:15:37.753988 28190 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3af9aa3f57c44431a8ebd226a45570f0 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:15:37.754395 28197 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3af9aa3f57c44431a8ebd226a45570f0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3af9aa3f57c44431a8ebd226a45570f0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3af9aa3f57c44431a8ebd226a45570f0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44993 } } }
I20260807 15:15:37.754633 28197 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3af9aa3f57c44431a8ebd226a45570f0 [sys.catalog]: This master's current role is: LEADER
I20260807 15:15:37.754560 28196 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3af9aa3f57c44431a8ebd226a45570f0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3af9aa3f57c44431a8ebd226a45570f0. Latest consensus state: current_term: 1 leader_uuid: "3af9aa3f57c44431a8ebd226a45570f0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3af9aa3f57c44431a8ebd226a45570f0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44993 } } }
I20260807 15:15:37.755024 28196 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3af9aa3f57c44431a8ebd226a45570f0 [sys.catalog]: This master's current role is: LEADER
I20260807 15:15:37.755193 28199 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:15:37.755728 28199 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:15:37.757396 28193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:37.746654 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53147 (local address 127.0.0.1:44993)
0807 15:15:37.746818 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:37.746826 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:37.746850 (+ 24us) server_negotiation.cc:407] Connection header received
0807 15:15:37.747263 (+ 413us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:37.747269 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:37.747313 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:37.747425 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:15:37.748237 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:37.749194 (+ 957us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:37.750196 (+ 1002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:37.750610 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:37.750699 (+ 89us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:37.753733 (+ 3034us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:15:37.753737 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:15:37.753737 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:15:37.753748 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:15:37.755632 (+ 1884us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:15:37.756166 (+ 534us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:15:37.756168 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:15:37.756168 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:15:37.756270 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:15:37.756507 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:15:37.756508 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:15:37.756508 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:15:37.756810 (+ 302us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:15:37.756964 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0807 15:15:37.757105 (+ 141us) server_negotiation.cc:299] Negotiation successful
0807 15:15:37.757208 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":118,"thread_start_us":69,"threads_started":1}
I20260807 15:15:37.757606 28199 catalog_manager.cc:1347] Generated new cluster ID: bf12f379c143436e947a2c4417ec3ad1
I20260807 15:15:37.757764 28199 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:15:37.758603 30472 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 15:15:37.768229 28193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:37.760021 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48555 (local address 127.0.0.1:44993)
0807 15:15:37.760053 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:37.760055 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:37.760170 (+ 115us) server_negotiation.cc:407] Connection header received
0807 15:15:37.760234 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:37.760235 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:37.760243 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:37.760467 (+ 224us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:15:37.761114 (+ 647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:37.762077 (+ 963us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:37.763116 (+ 1039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:37.763605 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:37.763684 (+ 79us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:37.765154 (+ 1470us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:15:37.765156 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:15:37.765156 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:15:37.765163 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:15:37.766707 (+ 1544us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:15:37.767330 (+ 623us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:15:37.767331 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:15:37.767332 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:15:37.767399 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:15:37.767628 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:15:37.767628 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:15:37.767628 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:15:37.767754 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:15:37.767861 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0807 15:15:37.767972 (+ 111us) server_negotiation.cc:299] Negotiation successful
0807 15:15:37.768040 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260807 15:15:37.783097 28199 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:15:37.783944 28199 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:15:37.792663 28199 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3af9aa3f57c44431a8ebd226a45570f0: Generated new TSK 0
I20260807 15:15:37.792878 28199 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 15:15:37.803858 30472 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7fZSIL/build/release/bin/kudu with pid 28082
W20260807 15:15:37.812991 28056 connection.cc:535] client connection to 127.0.0.1:44993 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:44993): Connection reset by peer (error 104)
2026-08-07T15:15:37Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1463 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786115480851507-30472-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 15:15:37 dist-test-slave-42z9 krb5kdc[28229](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 15:15:37 dist-test-slave-42z9 krb5kdc[28229](info): set up 2 sockets
Aug 07 15:15:37 dist-test-slave-42z9 krb5kdc[28229](info): commencing operation
krb5kdc: starting...
W20260807 15:15:38.934693 30472 mini_kdc.cc:121] Time spent starting KDC: real 1.108s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786115480851507-30472-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786115480851507-30472-0/krb5kdc/test-user.keytab.
Aug 07 15:15:39 dist-test-slave-42z9 krb5kdc[28229](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115739, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T15:15:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T15:15:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786115480851507-30472-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786115480851507-30472-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 15:15:39.065120 30472 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7fZSIL/build/release/bin/kudu
/tmp/dist-test-task7fZSIL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786115480851507-30472-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786115480851507-30472-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786115480851507-30472-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786115480851507-30472-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34751
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786115480851507-30472-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42773
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34751
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786115480851507-30472-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786115480851507-30472-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786115480851507-30472-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:15:39.149866 28251 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:15:39.150156 28251 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:15:39.150194 28251 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:15:39.151446 28251 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:15:39.151497 28251 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:15:39.151517 28251 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:15:39.151533 28251 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:15:39.151549 28251 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:15:39.153126 28251 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42773
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786115480851507-30472-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786115480851507-30472-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34751
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786115480851507-30472-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34751
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786115480851507-30472-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28251
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786115480851507-30472-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:15:39.153411 28251 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:15:39.153663 28251 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:15:39.158903 28261 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:15:39.159017 28251 server_base.cc:1034] running on GCE node
W20260807 15:15:39.159132 28262 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:15:39.159137 28266 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:15:39.159699 28251 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:15:39.160065 28251 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:15:39.161223 28251 hybrid_clock.cc:648] HybridClock initialized: now 1786115739161209 us; error 35 us; skew 500 ppm
Aug 07 15:15:39 dist-test-slave-42z9 krb5kdc[28229](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115739, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:15:39.164319 28251 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 15:15:39.164798 28251 webserver.cc:460] Webserver started at http://127.0.0.1:46525/ using document root <none> and password file <none>
I20260807 15:15:39.165025 28251 fs_manager.cc:359] Metadata directory not provided
I20260807 15:15:39.165071 28251 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:15:39.165205 28251 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:15:39.166380 28251 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786115480851507-30472-0/minicluster-data/master-0/data/instance:
uuid: "0a569acbbbb24e06b8d0df4b14f1fe22"
format_stamp: "Formatted at 2026-08-07 15:15:39 on dist-test-slave-42z9"
server_key: "265e3b1e81d662000c47b7f88ecf60b5"
server_key_iv: "b3244b280af2629e0c015903d67b99a5"
server_key_version: "encryptionkey@0"
I20260807 15:15:39.166741 28251 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786115480851507-30472-0/minicluster-data/master-0/wal/instance:
uuid: "0a569acbbbb24e06b8d0df4b14f1fe22"
format_stamp: "Formatted at 2026-08-07 15:15:39 on dist-test-slave-42z9"
server_key: "265e3b1e81d662000c47b7f88ecf60b5"
server_key_iv: "b3244b280af2629e0c015903d67b99a5"
server_key_version: "encryptionkey@0"
I20260807 15:15:39.168138 28251 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260807 15:15:39.168936 28273 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:15:39.169090 28251 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260807 15:15:39.169181 28251 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786115480851507-30472-0/minicluster-data/master-0/data,/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786115480851507-30472-0/minicluster-data/master-0/wal
uuid: "0a569acbbbb24e06b8d0df4b14f1fe22"
format_stamp: "Formatted at 2026-08-07 15:15:39 on dist-test-slave-42z9"
server_key: "265e3b1e81d662000c47b7f88ecf60b5"
server_key_iv: "b3244b280af2629e0c015903d67b99a5"
server_key_version: "encryptionkey@0"
I20260807 15:15:39.169248 28251 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786115480851507-30472-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786115480851507-30472-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786115480851507-30472-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:15:39.177206 28251 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:15:39.180330 28251 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:15:39.180497 28251 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:15:39.185052 28251 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34751
I20260807 15:15:39.185115 28352 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34751 every 8 connection(s)
I20260807 15:15:39.185603 28251 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786115480851507-30472-0/minicluster-data/master-0/data/info.pb
I20260807 15:15:39.186780 28354 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:15:39.189221 28354 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0a569acbbbb24e06b8d0df4b14f1fe22: Bootstrap starting.
I20260807 15:15:39.189878 28354 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0a569acbbbb24e06b8d0df4b14f1fe22: Neither blocks nor log segments found. Creating new log.
I20260807 15:15:39.190156 28354 log.cc:824] T 00000000000000000000000000000000 P 0a569acbbbb24e06b8d0df4b14f1fe22: Log is configured to *not* fsync() on all Append() calls
I20260807 15:15:39.190887 28354 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0a569acbbbb24e06b8d0df4b14f1fe22: No bootstrap required, opened a new log
I20260807 15:15:39.191195 30472 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7fZSIL/build/release/bin/kudu as pid 28251
I20260807 15:15:39.191345 30472 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786115480851507-30472-0/minicluster-data/master-0/wal/instance
I20260807 15:15:39.191540 30472 external_mini_cluster.cc:1428] Setting key 0c741134abfc482a266d9dd2a4e54a9f
I20260807 15:15:39.192800 28354 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0a569acbbbb24e06b8d0df4b14f1fe22 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a569acbbbb24e06b8d0df4b14f1fe22" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34751 } }
I20260807 15:15:39.192940 28354 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0a569acbbbb24e06b8d0df4b14f1fe22 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:15:39.192972 28354 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0a569acbbbb24e06b8d0df4b14f1fe22 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a569acbbbb24e06b8d0df4b14f1fe22, State: Initialized, Role: FOLLOWER
I20260807 15:15:39.193117 28354 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0a569acbbbb24e06b8d0df4b14f1fe22 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a569acbbbb24e06b8d0df4b14f1fe22" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34751 } }
I20260807 15:15:39.193193 28354 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0a569acbbbb24e06b8d0df4b14f1fe22 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:15:39.193225 28354 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0a569acbbbb24e06b8d0df4b14f1fe22 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:15:39.193274 28354 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0a569acbbbb24e06b8d0df4b14f1fe22 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:15:39.193912 28354 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0a569acbbbb24e06b8d0df4b14f1fe22 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a569acbbbb24e06b8d0df4b14f1fe22" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34751 } }
I20260807 15:15:39.194005 28354 leader_election.cc:302] T 00000000000000000000000000000000 P 0a569acbbbb24e06b8d0df4b14f1fe22 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0a569acbbbb24e06b8d0df4b14f1fe22; no voters:
I20260807 15:15:39.194159 28354 leader_election.cc:288] T 00000000000000000000000000000000 P 0a569acbbbb24e06b8d0df4b14f1fe22 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 15:15:39.194203 28360 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0a569acbbbb24e06b8d0df4b14f1fe22 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:15:39.194355 28360 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0a569acbbbb24e06b8d0df4b14f1fe22 [term 1 LEADER]: Becoming Leader. State: Replica: 0a569acbbbb24e06b8d0df4b14f1fe22, State: Running, Role: LEADER
I20260807 15:15:39.194439 28354 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0a569acbbbb24e06b8d0df4b14f1fe22 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:15:39.194530 28360 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0a569acbbbb24e06b8d0df4b14f1fe22 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a569acbbbb24e06b8d0df4b14f1fe22" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34751 } }
I20260807 15:15:39.194875 28360 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0a569acbbbb24e06b8d0df4b14f1fe22 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0a569acbbbb24e06b8d0df4b14f1fe22. Latest consensus state: current_term: 1 leader_uuid: "0a569acbbbb24e06b8d0df4b14f1fe22" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a569acbbbb24e06b8d0df4b14f1fe22" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34751 } } }
I20260807 15:15:39.195116 28360 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0a569acbbbb24e06b8d0df4b14f1fe22 [sys.catalog]: This master's current role is: LEADER
I20260807 15:15:39.195045 28362 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0a569acbbbb24e06b8d0df4b14f1fe22 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0a569acbbbb24e06b8d0df4b14f1fe22" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a569acbbbb24e06b8d0df4b14f1fe22" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34751 } } }
I20260807 15:15:39.195502 28362 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0a569acbbbb24e06b8d0df4b14f1fe22 [sys.catalog]: This master's current role is: LEADER
I20260807 15:15:39.195720 28374 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:15:39.196223 28374 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:15:39.198320 28359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:39.192142 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42457 (local address 127.0.0.1:34751)
0807 15:15:39.192309 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:39.192317 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:39.192344 (+ 27us) server_negotiation.cc:407] Connection header received
0807 15:15:39.192805 (+ 461us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:39.192814 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:39.192879 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:39.193014 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:15:39.193241 (+ 227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:39.194844 (+ 1603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:39.195997 (+ 1153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:39.196702 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:39.197177 (+ 475us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:39.197630 (+ 453us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:15:39.197634 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:15:39.197635 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:15:39.197836 (+ 201us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:15:39.197875 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:15:39.197892 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:15:39.197982 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0807 15:15:39.198081 (+ 99us) server_negotiation.cc:299] Negotiation successful
0807 15:15:39.198110 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":117,"thread_start_us":86,"threads_started":1}
I20260807 15:15:39.199298 30472 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 15:15:39.201920 28374 catalog_manager.cc:1347] Generated new cluster ID: b642c4340238470e9e856068e01500bd
I20260807 15:15:39.201995 28374 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:15:39.204437 28359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:39.200916 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60251 (local address 127.0.0.1:34751)
0807 15:15:39.200970 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:39.200972 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:39.201097 (+ 125us) server_negotiation.cc:407] Connection header received
0807 15:15:39.201169 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:39.201170 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:39.201180 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:39.201227 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:15:39.201363 (+ 136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:39.202005 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:39.203032 (+ 1027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:39.203304 (+ 272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:39.203374 (+ 70us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:39.204137 (+ 763us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:15:39.204138 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:15:39.204139 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:15:39.204154 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:15:39.204173 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:15:39.204178 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:15:39.204223 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0807 15:15:39.204301 (+ 78us) server_negotiation.cc:299] Negotiation successful
0807 15:15:39.204317 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260807 15:15:39.215008 28374 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:15:39.215674 28374 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:15:39.222885 28374 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0a569acbbbb24e06b8d0df4b14f1fe22: Generated new TSK 0
I20260807 15:15:39.223080 28374 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 15:15:39.228683 30472 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7fZSIL/build/release/bin/kudu with pid 28251
W20260807 15:15:39.250605 28227 connection.cc:535] client connection to 127.0.0.1:34751 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:34751): Connection reset by peer (error 104)
2026-08-07T15:15:39Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1437 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786115480851507-30472-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 15:15:39 dist-test-slave-42z9 krb5kdc[28401](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 15:15:39 dist-test-slave-42z9 krb5kdc[28401](info): set up 2 sockets
Aug 07 15:15:39 dist-test-slave-42z9 krb5kdc[28401](info): commencing operation
krb5kdc: starting...
W20260807 15:15:40.373646 30472 mini_kdc.cc:121] Time spent starting KDC: real 1.109s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786115480851507-30472-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786115480851507-30472-0/krb5kdc/test-user.keytab.
Aug 07 15:15:40 dist-test-slave-42z9 krb5kdc[28401](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115740, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T15:15:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T15:15:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786115480851507-30472-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786115480851507-30472-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 15:15:40.508981 30472 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7fZSIL/build/release/bin/kudu
/tmp/dist-test-task7fZSIL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786115480851507-30472-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786115480851507-30472-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786115480851507-30472-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786115480851507-30472-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38857
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786115480851507-30472-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40989
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38857
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786115480851507-30472-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786115480851507-30472-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786115480851507-30472-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:15:40.594787 28418 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:15:40.595021 28418 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:15:40.595046 28418 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:15:40.596220 28418 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:15:40.596269 28418 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:15:40.596282 28418 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:15:40.596294 28418 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:15:40.596305 28418 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:15:40.597926 28418 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40989
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786115480851507-30472-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786115480851507-30472-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38857
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786115480851507-30472-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38857
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786115480851507-30472-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28418
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786115480851507-30472-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:15:40.598189 28418 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:15:40.598418 28418 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:15:40.603457 28429 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:15:40.603627 28427 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:15:40.603713 28426 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:15:40.603847 28418 server_base.cc:1034] running on GCE node
I20260807 15:15:40.604123 28418 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:15:40.604424 28418 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:15:40.605582 28418 hybrid_clock.cc:648] HybridClock initialized: now 1786115740605563 us; error 37 us; skew 500 ppm
Aug 07 15:15:40 dist-test-slave-42z9 krb5kdc[28401](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115740, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:15:40.608579 28418 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 15:15:40.609421 28418 webserver.cc:460] Webserver started at http://127.0.0.1:38169/ using document root <none> and password file <none>
I20260807 15:15:40.609721 28418 fs_manager.cc:359] Metadata directory not provided
I20260807 15:15:40.609774 28418 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:15:40.609908 28418 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:15:40.610999 28418 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786115480851507-30472-0/minicluster-data/master-0/data/instance:
uuid: "bf3f59ecab1a4fdf9a817f4b0f51ace1"
format_stamp: "Formatted at 2026-08-07 15:15:40 on dist-test-slave-42z9"
server_key: "c950e8df0eb113107538b062e563a65f"
server_key_iv: "ec6469fc5229d2450b10f31e6c872fd4"
server_key_version: "encryptionkey@0"
I20260807 15:15:40.611438 28418 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786115480851507-30472-0/minicluster-data/master-0/wal/instance:
uuid: "bf3f59ecab1a4fdf9a817f4b0f51ace1"
format_stamp: "Formatted at 2026-08-07 15:15:40 on dist-test-slave-42z9"
server_key: "c950e8df0eb113107538b062e563a65f"
server_key_iv: "ec6469fc5229d2450b10f31e6c872fd4"
server_key_version: "encryptionkey@0"
I20260807 15:15:40.612804 28418 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260807 15:15:40.613734 28440 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:15:40.613921 28418 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 15:15:40.614017 28418 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786115480851507-30472-0/minicluster-data/master-0/data,/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786115480851507-30472-0/minicluster-data/master-0/wal
uuid: "bf3f59ecab1a4fdf9a817f4b0f51ace1"
format_stamp: "Formatted at 2026-08-07 15:15:40 on dist-test-slave-42z9"
server_key: "c950e8df0eb113107538b062e563a65f"
server_key_iv: "ec6469fc5229d2450b10f31e6c872fd4"
server_key_version: "encryptionkey@0"
I20260807 15:15:40.614084 28418 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786115480851507-30472-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786115480851507-30472-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786115480851507-30472-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:15:40.617501 28418 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:15:40.619732 28418 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:15:40.619874 28418 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:15:40.624251 28418 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38857
I20260807 15:15:40.624344 28525 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38857 every 8 connection(s)
I20260807 15:15:40.624661 28418 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786115480851507-30472-0/minicluster-data/master-0/data/info.pb
I20260807 15:15:40.625221 30472 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7fZSIL/build/release/bin/kudu as pid 28418
I20260807 15:15:40.625330 30472 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786115480851507-30472-0/minicluster-data/master-0/wal/instance
I20260807 15:15:40.625548 30472 external_mini_cluster.cc:1428] Setting key e37ac2f5249b393a5f129a48cf498c75
I20260807 15:15:40.625866 28526 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:15:40.628532 28529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:40.626095 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34783 (local address 127.0.0.1:38857)
0807 15:15:40.626307 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:40.626316 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:40.626359 (+ 43us) server_negotiation.cc:407] Connection header received
0807 15:15:40.627024 (+ 665us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:40.627034 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:40.627069 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:40.627235 (+ 166us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:15:40.627902 (+ 667us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:15:40.627907 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:15:40.627908 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:15:40.627913 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:15:40.627961 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:15:40.627975 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:15:40.628055 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0807 15:15:40.628287 (+ 232us) server_negotiation.cc:299] Negotiation successful
0807 15:15:40.628329 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":154,"thread_start_us":86,"threads_started":1}
I20260807 15:15:40.629235 28526 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bf3f59ecab1a4fdf9a817f4b0f51ace1: Bootstrap starting.
I20260807 15:15:40.629776 30472 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 15:15:40.629982 28526 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bf3f59ecab1a4fdf9a817f4b0f51ace1: Neither blocks nor log segments found. Creating new log.
I20260807 15:15:40.630261 28526 log.cc:824] T 00000000000000000000000000000000 P bf3f59ecab1a4fdf9a817f4b0f51ace1: Log is configured to *not* fsync() on all Append() calls
I20260807 15:15:40.631100 28526 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bf3f59ecab1a4fdf9a817f4b0f51ace1: No bootstrap required, opened a new log
I20260807 15:15:40.632347 28529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:40.631444 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40123 (local address 127.0.0.1:38857)
0807 15:15:40.631507 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:40.631509 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:40.631517 (+ 8us) server_negotiation.cc:407] Connection header received
0807 15:15:40.631562 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:40.631563 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:40.631567 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:40.631620 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:15:40.632149 (+ 529us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:15:40.632150 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:15:40.632151 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:15:40.632155 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:15:40.632170 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:15:40.632175 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:15:40.632221 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0807 15:15:40.632273 (+ 52us) server_negotiation.cc:299] Negotiation successful
0807 15:15:40.632286 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260807 15:15:40.633592 28526 raft_consensus.cc:348] T 00000000000000000000000000000000 P bf3f59ecab1a4fdf9a817f4b0f51ace1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf3f59ecab1a4fdf9a817f4b0f51ace1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38857 } }
I20260807 15:15:40.633787 28526 raft_consensus.cc:374] T 00000000000000000000000000000000 P bf3f59ecab1a4fdf9a817f4b0f51ace1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:15:40.633814 28526 raft_consensus.cc:729] T 00000000000000000000000000000000 P bf3f59ecab1a4fdf9a817f4b0f51ace1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bf3f59ecab1a4fdf9a817f4b0f51ace1, State: Initialized, Role: FOLLOWER
I20260807 15:15:40.633915 28526 consensus_queue.cc:260] T 00000000000000000000000000000000 P bf3f59ecab1a4fdf9a817f4b0f51ace1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf3f59ecab1a4fdf9a817f4b0f51ace1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38857 } }
I20260807 15:15:40.634011 28526 raft_consensus.cc:388] T 00000000000000000000000000000000 P bf3f59ecab1a4fdf9a817f4b0f51ace1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:15:40.634056 28526 raft_consensus.cc:482] T 00000000000000000000000000000000 P bf3f59ecab1a4fdf9a817f4b0f51ace1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:15:40.634105 28526 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bf3f59ecab1a4fdf9a817f4b0f51ace1 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:15:40.634755 28526 raft_consensus.cc:504] T 00000000000000000000000000000000 P bf3f59ecab1a4fdf9a817f4b0f51ace1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf3f59ecab1a4fdf9a817f4b0f51ace1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38857 } }
I20260807 15:15:40.634846 28526 leader_election.cc:302] T 00000000000000000000000000000000 P bf3f59ecab1a4fdf9a817f4b0f51ace1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bf3f59ecab1a4fdf9a817f4b0f51ace1; no voters:
I20260807 15:15:40.634976 28526 leader_election.cc:288] T 00000000000000000000000000000000 P bf3f59ecab1a4fdf9a817f4b0f51ace1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 15:15:40.635033 28540 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bf3f59ecab1a4fdf9a817f4b0f51ace1 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:15:40.635213 28526 sys_catalog.cc:564] T 00000000000000000000000000000000 P bf3f59ecab1a4fdf9a817f4b0f51ace1 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:15:40.635241 28540 raft_consensus.cc:686] T 00000000000000000000000000000000 P bf3f59ecab1a4fdf9a817f4b0f51ace1 [term 1 LEADER]: Becoming Leader. State: Replica: bf3f59ecab1a4fdf9a817f4b0f51ace1, State: Running, Role: LEADER
I20260807 15:15:40.635377 28540 consensus_queue.cc:237] T 00000000000000000000000000000000 P bf3f59ecab1a4fdf9a817f4b0f51ace1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf3f59ecab1a4fdf9a817f4b0f51ace1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38857 } }
I20260807 15:15:40.635748 28544 sys_catalog.cc:455] T 00000000000000000000000000000000 P bf3f59ecab1a4fdf9a817f4b0f51ace1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bf3f59ecab1a4fdf9a817f4b0f51ace1. Latest consensus state: current_term: 1 leader_uuid: "bf3f59ecab1a4fdf9a817f4b0f51ace1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf3f59ecab1a4fdf9a817f4b0f51ace1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38857 } } }
I20260807 15:15:40.635751 28542 sys_catalog.cc:455] T 00000000000000000000000000000000 P bf3f59ecab1a4fdf9a817f4b0f51ace1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bf3f59ecab1a4fdf9a817f4b0f51ace1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf3f59ecab1a4fdf9a817f4b0f51ace1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38857 } } }
I20260807 15:15:40.635921 28544 sys_catalog.cc:458] T 00000000000000000000000000000000 P bf3f59ecab1a4fdf9a817f4b0f51ace1 [sys.catalog]: This master's current role is: LEADER
I20260807 15:15:40.635999 28542 sys_catalog.cc:458] T 00000000000000000000000000000000 P bf3f59ecab1a4fdf9a817f4b0f51ace1 [sys.catalog]: This master's current role is: LEADER
I20260807 15:15:40.636410 28553 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:15:40.636915 28553 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:15:40.638319 28553 catalog_manager.cc:1347] Generated new cluster ID: 12b20fd69713449387766c27a85949a1
I20260807 15:15:40.638373 28553 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:15:40.647079 28553 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:15:40.647270 28553 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:15:40.652427 28553 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bf3f59ecab1a4fdf9a817f4b0f51ace1: Generated new TSK 0
I20260807 15:15:40.652621 28553 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 15:15:40.657917 30472 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7fZSIL/build/release/bin/kudu with pid 28418
2026-08-07T15:15:40Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1418 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786115480851507-30472-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 15:15:40 dist-test-slave-42z9 krb5kdc[28568](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 15:15:40 dist-test-slave-42z9 krb5kdc[28568](info): set up 2 sockets
Aug 07 15:15:40 dist-test-slave-42z9 krb5kdc[28568](info): commencing operation
krb5kdc: starting...
W20260807 15:15:41.803228 30472 mini_kdc.cc:121] Time spent starting KDC: real 1.120s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786115480851507-30472-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786115480851507-30472-0/krb5kdc/test-user.keytab.
Aug 07 15:15:41 dist-test-slave-42z9 krb5kdc[28568](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115741, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T15:15:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T15:15:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786115480851507-30472-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786115480851507-30472-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 15:15:41.936530 30472 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7fZSIL/build/release/bin/kudu
/tmp/dist-test-task7fZSIL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786115480851507-30472-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786115480851507-30472-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786115480851507-30472-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786115480851507-30472-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44575
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786115480851507-30472-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46045
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44575
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786115480851507-30472-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786115480851507-30472-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786115480851507-30472-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:15:42.024307 28594 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:15:42.024570 28594 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:15:42.024598 28594 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:15:42.025890 28594 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:15:42.025944 28594 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:15:42.025957 28594 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:15:42.025969 28594 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:15:42.025979 28594 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:15:42.027586 28594 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46045
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786115480851507-30472-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786115480851507-30472-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44575
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786115480851507-30472-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44575
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786115480851507-30472-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28594
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786115480851507-30472-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:15:42.027845 28594 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:15:42.028079 28594 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:15:42.033242 28601 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:15:42.033372 28594 server_base.cc:1034] running on GCE node
W20260807 15:15:42.033270 28603 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:15:42.033270 28600 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:15:42.033792 28594 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:15:42.034127 28594 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:15:42.035283 28594 hybrid_clock.cc:648] HybridClock initialized: now 1786115742035279 us; error 52 us; skew 500 ppm
Aug 07 15:15:42 dist-test-slave-42z9 krb5kdc[28568](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115742, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:15:42.037945 28594 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 15:15:42.038445 28594 webserver.cc:460] Webserver started at http://127.0.0.1:39543/ using document root <none> and password file <none>
I20260807 15:15:42.038684 28594 fs_manager.cc:359] Metadata directory not provided
I20260807 15:15:42.038735 28594 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:15:42.038864 28594 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:15:42.039961 28594 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786115480851507-30472-0/minicluster-data/master-0/data/instance:
uuid: "ea57186bca1c4779a0f5c17cc7e1fa6d"
format_stamp: "Formatted at 2026-08-07 15:15:42 on dist-test-slave-42z9"
server_key: "7f819f5235a54f4814ffa16f7e6fc653"
server_key_iv: "93d862c911557bac63d8514b507cd40c"
server_key_version: "encryptionkey@0"
I20260807 15:15:42.040306 28594 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786115480851507-30472-0/minicluster-data/master-0/wal/instance:
uuid: "ea57186bca1c4779a0f5c17cc7e1fa6d"
format_stamp: "Formatted at 2026-08-07 15:15:42 on dist-test-slave-42z9"
server_key: "7f819f5235a54f4814ffa16f7e6fc653"
server_key_iv: "93d862c911557bac63d8514b507cd40c"
server_key_version: "encryptionkey@0"
I20260807 15:15:42.041738 28594 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260807 15:15:42.042505 28619 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:15:42.042657 28594 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 15:15:42.042743 28594 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786115480851507-30472-0/minicluster-data/master-0/data,/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786115480851507-30472-0/minicluster-data/master-0/wal
uuid: "ea57186bca1c4779a0f5c17cc7e1fa6d"
format_stamp: "Formatted at 2026-08-07 15:15:42 on dist-test-slave-42z9"
server_key: "7f819f5235a54f4814ffa16f7e6fc653"
server_key_iv: "93d862c911557bac63d8514b507cd40c"
server_key_version: "encryptionkey@0"
I20260807 15:15:42.042805 28594 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786115480851507-30472-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786115480851507-30472-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786115480851507-30472-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:15:42.057641 28594 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:15:42.059773 28594 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:15:42.059928 28594 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:15:42.064729 28594 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44575
I20260807 15:15:42.064824 28695 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44575 every 8 connection(s)
I20260807 15:15:42.065147 28594 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786115480851507-30472-0/minicluster-data/master-0/data/info.pb
I20260807 15:15:42.066598 28699 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:15:42.070541 28699 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ea57186bca1c4779a0f5c17cc7e1fa6d: Bootstrap starting.
I20260807 15:15:42.071216 28699 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ea57186bca1c4779a0f5c17cc7e1fa6d: Neither blocks nor log segments found. Creating new log.
I20260807 15:15:42.071516 28699 log.cc:824] T 00000000000000000000000000000000 P ea57186bca1c4779a0f5c17cc7e1fa6d: Log is configured to *not* fsync() on all Append() calls
I20260807 15:15:42.072456 28699 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ea57186bca1c4779a0f5c17cc7e1fa6d: No bootstrap required, opened a new log
I20260807 15:15:42.072750 30472 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7fZSIL/build/release/bin/kudu as pid 28594
I20260807 15:15:42.072861 30472 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786115480851507-30472-0/minicluster-data/master-0/wal/instance
I20260807 15:15:42.073051 30472 external_mini_cluster.cc:1428] Setting key 55abb5781f8f65623ed58b455445ec79
I20260807 15:15:42.074869 28699 raft_consensus.cc:348] T 00000000000000000000000000000000 P ea57186bca1c4779a0f5c17cc7e1fa6d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea57186bca1c4779a0f5c17cc7e1fa6d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44575 } }
I20260807 15:15:42.075030 28699 raft_consensus.cc:374] T 00000000000000000000000000000000 P ea57186bca1c4779a0f5c17cc7e1fa6d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:15:42.075057 28699 raft_consensus.cc:729] T 00000000000000000000000000000000 P ea57186bca1c4779a0f5c17cc7e1fa6d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ea57186bca1c4779a0f5c17cc7e1fa6d, State: Initialized, Role: FOLLOWER
I20260807 15:15:42.075210 28699 consensus_queue.cc:260] T 00000000000000000000000000000000 P ea57186bca1c4779a0f5c17cc7e1fa6d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea57186bca1c4779a0f5c17cc7e1fa6d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44575 } }
I20260807 15:15:42.075299 28699 raft_consensus.cc:388] T 00000000000000000000000000000000 P ea57186bca1c4779a0f5c17cc7e1fa6d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:15:42.075332 28699 raft_consensus.cc:482] T 00000000000000000000000000000000 P ea57186bca1c4779a0f5c17cc7e1fa6d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:15:42.075371 28699 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ea57186bca1c4779a0f5c17cc7e1fa6d [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:15:42.076520 28699 raft_consensus.cc:504] T 00000000000000000000000000000000 P ea57186bca1c4779a0f5c17cc7e1fa6d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea57186bca1c4779a0f5c17cc7e1fa6d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44575 } }
I20260807 15:15:42.076622 28699 leader_election.cc:302] T 00000000000000000000000000000000 P ea57186bca1c4779a0f5c17cc7e1fa6d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ea57186bca1c4779a0f5c17cc7e1fa6d; no voters:
I20260807 15:15:42.076756 28699 leader_election.cc:288] T 00000000000000000000000000000000 P ea57186bca1c4779a0f5c17cc7e1fa6d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 15:15:42.076859 28705 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ea57186bca1c4779a0f5c17cc7e1fa6d [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:15:42.077059 28705 raft_consensus.cc:686] T 00000000000000000000000000000000 P ea57186bca1c4779a0f5c17cc7e1fa6d [term 1 LEADER]: Becoming Leader. State: Replica: ea57186bca1c4779a0f5c17cc7e1fa6d, State: Running, Role: LEADER
I20260807 15:15:42.077071 28699 sys_catalog.cc:564] T 00000000000000000000000000000000 P ea57186bca1c4779a0f5c17cc7e1fa6d [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:15:42.077201 28705 consensus_queue.cc:237] T 00000000000000000000000000000000 P ea57186bca1c4779a0f5c17cc7e1fa6d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea57186bca1c4779a0f5c17cc7e1fa6d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44575 } }
I20260807 15:15:42.077630 28711 sys_catalog.cc:455] T 00000000000000000000000000000000 P ea57186bca1c4779a0f5c17cc7e1fa6d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ea57186bca1c4779a0f5c17cc7e1fa6d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea57186bca1c4779a0f5c17cc7e1fa6d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44575 } } }
I20260807 15:15:42.077716 28711 sys_catalog.cc:458] T 00000000000000000000000000000000 P ea57186bca1c4779a0f5c17cc7e1fa6d [sys.catalog]: This master's current role is: LEADER
I20260807 15:15:42.077915 28712 sys_catalog.cc:455] T 00000000000000000000000000000000 P ea57186bca1c4779a0f5c17cc7e1fa6d [sys.catalog]: SysCatalogTable state changed. Reason: New leader ea57186bca1c4779a0f5c17cc7e1fa6d. Latest consensus state: current_term: 1 leader_uuid: "ea57186bca1c4779a0f5c17cc7e1fa6d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea57186bca1c4779a0f5c17cc7e1fa6d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44575 } } }
I20260807 15:15:42.078027 28712 sys_catalog.cc:458] T 00000000000000000000000000000000 P ea57186bca1c4779a0f5c17cc7e1fa6d [sys.catalog]: This master's current role is: LEADER
I20260807 15:15:42.078413 28720 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:15:42.079042 28720 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:15:42.080698 28720 catalog_manager.cc:1347] Generated new cluster ID: c1bae3352a894cdc91234f618e25bac6
I20260807 15:15:42.080745 28720 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 07 15:15:42 dist-test-slave-42z9 krb5kdc[28568](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115741, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260807 15:15:42.086207 28704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:42.073689 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56883 (local address 127.0.0.1:44575)
0807 15:15:42.073916 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:42.073925 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:42.074059 (+ 134us) server_negotiation.cc:407] Connection header received
0807 15:15:42.074871 (+ 812us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:42.074880 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:42.074939 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:42.075104 (+ 165us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:15:42.075973 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:42.077514 (+ 1541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:42.078594 (+ 1080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:42.079110 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:42.079256 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:42.082098 (+ 2842us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:15:42.082102 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:15:42.082102 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:15:42.082115 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:15:42.084106 (+ 1991us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:15:42.084531 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:15:42.084534 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:15:42.084534 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:15:42.085009 (+ 475us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:15:42.085225 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:15:42.085226 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:15:42.085226 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:15:42.085574 (+ 348us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:15:42.085725 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0807 15:15:42.085837 (+ 112us) server_negotiation.cc:299] Negotiation successful
0807 15:15:42.085988 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":158,"thread_start_us":106,"threads_started":1}
I20260807 15:15:42.087543 30472 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 15:15:42.096057 28704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:42.089131 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50257 (local address 127.0.0.1:44575)
0807 15:15:42.089163 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:42.089165 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:42.089172 (+ 7us) server_negotiation.cc:407] Connection header received
0807 15:15:42.089227 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:42.089228 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:42.089238 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:42.089287 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:15:42.089932 (+ 645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:42.090669 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:42.091688 (+ 1019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:42.092032 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:42.092085 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:42.093421 (+ 1336us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:15:42.093423 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:15:42.093423 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:15:42.093432 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:15:42.094947 (+ 1515us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:15:42.095329 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:15:42.095330 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:15:42.095331 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:15:42.095382 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:15:42.095564 (+ 182us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:15:42.095565 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:15:42.095565 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:15:42.095681 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:15:42.095776 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0807 15:15:42.095880 (+ 104us) server_negotiation.cc:299] Negotiation successful
0807 15:15:42.095963 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260807 15:15:42.113148 28720 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:15:42.113773 28720 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:15:42.120728 28720 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ea57186bca1c4779a0f5c17cc7e1fa6d: Generated new TSK 0
I20260807 15:15:42.120929 28720 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 15:15:42.125273 30472 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7fZSIL/build/release/bin/kudu with pid 28594
2026-08-07T15:15:42Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1470 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786115480851507-30472-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 15:15:42 dist-test-slave-42z9 krb5kdc[28752](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 15:15:42 dist-test-slave-42z9 krb5kdc[28752](info): set up 2 sockets
Aug 07 15:15:42 dist-test-slave-42z9 krb5kdc[28752](info): commencing operation
krb5kdc: starting...
W20260807 15:15:43.271610 30472 mini_kdc.cc:121] Time spent starting KDC: real 1.118s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786115480851507-30472-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786115480851507-30472-0/krb5kdc/test-user.keytab.
Aug 07 15:15:43 dist-test-slave-42z9 krb5kdc[28752](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115743, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T15:15:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T15:15:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786115480851507-30472-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786115480851507-30472-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 15:15:43.402352 30472 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7fZSIL/build/release/bin/kudu
/tmp/dist-test-task7fZSIL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786115480851507-30472-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786115480851507-30472-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786115480851507-30472-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786115480851507-30472-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34411
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786115480851507-30472-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33583
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34411
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786115480851507-30472-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786115480851507-30472-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786115480851507-30472-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:15:43.486521 28773 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:15:43.486753 28773 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:15:43.486786 28773 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:15:43.487921 28773 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:15:43.487965 28773 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:15:43.487983 28773 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:15:43.487999 28773 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:15:43.488016 28773 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:15:43.489552 28773 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33583
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786115480851507-30472-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786115480851507-30472-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34411
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786115480851507-30472-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34411
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786115480851507-30472-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28773
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786115480851507-30472-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:15:43.489794 28773 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:15:43.490001 28773 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:15:43.494917 28782 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:15:43.494938 28785 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:15:43.494932 28783 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:15:43.495144 28773 server_base.cc:1034] running on GCE node
I20260807 15:15:43.495419 28773 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:15:43.495759 28773 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:15:43.496948 28773 hybrid_clock.cc:648] HybridClock initialized: now 1786115743496929 us; error 36 us; skew 500 ppm
Aug 07 15:15:43 dist-test-slave-42z9 krb5kdc[28752](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115743, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:15:43.503151 28773 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 15:15:43.503681 28773 webserver.cc:460] Webserver started at http://127.0.0.1:32809/ using document root <none> and password file <none>
I20260807 15:15:43.503917 28773 fs_manager.cc:359] Metadata directory not provided
I20260807 15:15:43.503966 28773 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:15:43.504078 28773 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:15:43.505137 28773 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786115480851507-30472-0/minicluster-data/master-0/data/instance:
uuid: "4fd3e80a808740e3a77c1fdcc2333e78"
format_stamp: "Formatted at 2026-08-07 15:15:43 on dist-test-slave-42z9"
server_key: "c17bde623c2234f9f3fb50f24cd0253b"
server_key_iv: "05d25649112c305a4c6622ce22130e15"
server_key_version: "encryptionkey@0"
I20260807 15:15:43.505748 28773 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786115480851507-30472-0/minicluster-data/master-0/wal/instance:
uuid: "4fd3e80a808740e3a77c1fdcc2333e78"
format_stamp: "Formatted at 2026-08-07 15:15:43 on dist-test-slave-42z9"
server_key: "c17bde623c2234f9f3fb50f24cd0253b"
server_key_iv: "05d25649112c305a4c6622ce22130e15"
server_key_version: "encryptionkey@0"
I20260807 15:15:43.507119 28773 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:15:43.507915 28796 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:15:43.508108 28773 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:15:43.508198 28773 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786115480851507-30472-0/minicluster-data/master-0/data,/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786115480851507-30472-0/minicluster-data/master-0/wal
uuid: "4fd3e80a808740e3a77c1fdcc2333e78"
format_stamp: "Formatted at 2026-08-07 15:15:43 on dist-test-slave-42z9"
server_key: "c17bde623c2234f9f3fb50f24cd0253b"
server_key_iv: "05d25649112c305a4c6622ce22130e15"
server_key_version: "encryptionkey@0"
I20260807 15:15:43.508273 28773 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786115480851507-30472-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786115480851507-30472-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786115480851507-30472-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:15:43.530483 28773 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:15:43.532796 28773 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:15:43.532932 28773 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:15:43.537271 28773 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34411
I20260807 15:15:43.537343 28870 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34411 every 8 connection(s)
I20260807 15:15:43.537670 28773 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786115480851507-30472-0/minicluster-data/master-0/data/info.pb
I20260807 15:15:43.538380 30472 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7fZSIL/build/release/bin/kudu as pid 28773
I20260807 15:15:43.538481 30472 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786115480851507-30472-0/minicluster-data/master-0/wal/instance
I20260807 15:15:43.538612 28872 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:15:43.538651 30472 external_mini_cluster.cc:1428] Setting key eb51f44816081ed3d9d17ad866fa0f11
I20260807 15:15:43.542040 28872 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4fd3e80a808740e3a77c1fdcc2333e78: Bootstrap starting.
I20260807 15:15:43.542755 28872 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4fd3e80a808740e3a77c1fdcc2333e78: Neither blocks nor log segments found. Creating new log.
I20260807 15:15:43.543099 28872 log.cc:824] T 00000000000000000000000000000000 P 4fd3e80a808740e3a77c1fdcc2333e78: Log is configured to *not* fsync() on all Append() calls
I20260807 15:15:43.543869 28872 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4fd3e80a808740e3a77c1fdcc2333e78: No bootstrap required, opened a new log
I20260807 15:15:43.544299 28878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:43.539159 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41861 (local address 127.0.0.1:34411)
0807 15:15:43.539362 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:43.539372 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:43.539409 (+ 37us) server_negotiation.cc:407] Connection header received
0807 15:15:43.539812 (+ 403us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:43.539821 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:43.539883 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:43.540031 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:15:43.540191 (+ 160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:43.541557 (+ 1366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:43.542287 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:43.542887 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:43.543013 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:43.543664 (+ 651us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:15:43.543668 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:15:43.543669 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:15:43.543827 (+ 158us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:15:43.543858 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:15:43.543868 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:15:43.543920 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0807 15:15:43.544025 (+ 105us) server_negotiation.cc:299] Negotiation successful
0807 15:15:43.544093 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":136,"thread_start_us":85,"threads_started":1}
I20260807 15:15:43.545481 30472 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 15:15:43.546020 28872 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4fd3e80a808740e3a77c1fdcc2333e78 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4fd3e80a808740e3a77c1fdcc2333e78" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34411 } }
I20260807 15:15:43.546154 28872 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4fd3e80a808740e3a77c1fdcc2333e78 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:15:43.546176 28872 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4fd3e80a808740e3a77c1fdcc2333e78 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4fd3e80a808740e3a77c1fdcc2333e78, State: Initialized, Role: FOLLOWER
I20260807 15:15:43.546285 28872 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4fd3e80a808740e3a77c1fdcc2333e78 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4fd3e80a808740e3a77c1fdcc2333e78" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34411 } }
I20260807 15:15:43.546439 28872 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4fd3e80a808740e3a77c1fdcc2333e78 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:15:43.546504 28872 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4fd3e80a808740e3a77c1fdcc2333e78 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:15:43.546554 28872 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4fd3e80a808740e3a77c1fdcc2333e78 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:15:43.547264 28872 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4fd3e80a808740e3a77c1fdcc2333e78 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4fd3e80a808740e3a77c1fdcc2333e78" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34411 } }
I20260807 15:15:43.547384 28872 leader_election.cc:302] T 00000000000000000000000000000000 P 4fd3e80a808740e3a77c1fdcc2333e78 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4fd3e80a808740e3a77c1fdcc2333e78; no voters:
I20260807 15:15:43.547546 28872 leader_election.cc:288] T 00000000000000000000000000000000 P 4fd3e80a808740e3a77c1fdcc2333e78 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 15:15:43.547600 28889 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4fd3e80a808740e3a77c1fdcc2333e78 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:15:43.547768 28872 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4fd3e80a808740e3a77c1fdcc2333e78 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:15:43.547791 28889 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4fd3e80a808740e3a77c1fdcc2333e78 [term 1 LEADER]: Becoming Leader. State: Replica: 4fd3e80a808740e3a77c1fdcc2333e78, State: Running, Role: LEADER
I20260807 15:15:43.547948 28889 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4fd3e80a808740e3a77c1fdcc2333e78 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4fd3e80a808740e3a77c1fdcc2333e78" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34411 } }
I20260807 15:15:43.548288 28890 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4fd3e80a808740e3a77c1fdcc2333e78 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4fd3e80a808740e3a77c1fdcc2333e78" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4fd3e80a808740e3a77c1fdcc2333e78" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34411 } } }
I20260807 15:15:43.548362 28890 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4fd3e80a808740e3a77c1fdcc2333e78 [sys.catalog]: This master's current role is: LEADER
I20260807 15:15:43.548581 28891 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4fd3e80a808740e3a77c1fdcc2333e78 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4fd3e80a808740e3a77c1fdcc2333e78. Latest consensus state: current_term: 1 leader_uuid: "4fd3e80a808740e3a77c1fdcc2333e78" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4fd3e80a808740e3a77c1fdcc2333e78" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34411 } } }
I20260807 15:15:43.548844 28891 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4fd3e80a808740e3a77c1fdcc2333e78 [sys.catalog]: This master's current role is: LEADER
I20260807 15:15:43.549027 28903 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:15:43.549599 28903 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:15:43.550177 28878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:43.546828 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32929 (local address 127.0.0.1:34411)
0807 15:15:43.546884 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:43.546885 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:43.547004 (+ 119us) server_negotiation.cc:407] Connection header received
0807 15:15:43.547027 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:43.547028 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:43.547040 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:43.547099 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:15:43.547183 (+ 84us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:43.547838 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:43.548832 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:43.549104 (+ 272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:43.549161 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:43.549865 (+ 704us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:15:43.549866 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:15:43.549876 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:15:43.549881 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:15:43.549894 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:15:43.549898 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:15:43.549940 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0807 15:15:43.550082 (+ 142us) server_negotiation.cc:299] Negotiation successful
0807 15:15:43.550102 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260807 15:15:43.550953 28903 catalog_manager.cc:1347] Generated new cluster ID: fb91926df1a6480eb30715ac0a6af174
I20260807 15:15:43.551007 28903 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:15:43.570009 28903 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:15:43.570812 28903 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:15:43.584450 28903 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4fd3e80a808740e3a77c1fdcc2333e78: Generated new TSK 0
I20260807 15:15:43.584673 28903 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 15:15:43.589061 30472 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7fZSIL/build/release/bin/kudu with pid 28773
2026-08-07T15:15:43Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1462 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786115480851507-30472-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 15:15:43 dist-test-slave-42z9 krb5kdc[28920](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 15:15:43 dist-test-slave-42z9 krb5kdc[28920](info): set up 2 sockets
Aug 07 15:15:43 dist-test-slave-42z9 krb5kdc[28920](info): commencing operation
krb5kdc: starting...
W20260807 15:15:44.738420 30472 mini_kdc.cc:121] Time spent starting KDC: real 1.122s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786115480851507-30472-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786115480851507-30472-0/krb5kdc/test-user.keytab.
Aug 07 15:15:44 dist-test-slave-42z9 krb5kdc[28920](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115744, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T15:15:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T15:15:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786115480851507-30472-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786115480851507-30472-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 15:15:44.864315 30472 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7fZSIL/build/release/bin/kudu
/tmp/dist-test-task7fZSIL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786115480851507-30472-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786115480851507-30472-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786115480851507-30472-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786115480851507-30472-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43291
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786115480851507-30472-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35835
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43291
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786115480851507-30472-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786115480851507-30472-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786115480851507-30472-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:15:44.953238 28940 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:15:44.953521 28940 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:15:44.953559 28940 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:15:44.954797 28940 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:15:44.954844 28940 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:15:44.954864 28940 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:15:44.954880 28940 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:15:44.954897 28940 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:15:44.956494 28940 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35835
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786115480851507-30472-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786115480851507-30472-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43291
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786115480851507-30472-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43291
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786115480851507-30472-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28940
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786115480851507-30472-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:15:44.956753 28940 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:15:44.956970 28940 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:15:44.961966 28948 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:15:44.961970 28947 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:15:44.961966 28952 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:15:44.962255 28940 server_base.cc:1034] running on GCE node
I20260807 15:15:44.962568 28940 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:15:44.962869 28940 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:15:44.964043 28940 hybrid_clock.cc:648] HybridClock initialized: now 1786115744964028 us; error 31 us; skew 500 ppm
Aug 07 15:15:44 dist-test-slave-42z9 krb5kdc[28920](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115744, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:15:44.967058 28940 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 15:15:44.967603 28940 webserver.cc:460] Webserver started at http://127.0.0.1:38069/ using document root <none> and password file <none>
I20260807 15:15:44.967814 28940 fs_manager.cc:359] Metadata directory not provided
I20260807 15:15:44.967861 28940 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:15:44.967995 28940 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:15:44.969082 28940 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786115480851507-30472-0/minicluster-data/master-0/data/instance:
uuid: "b96d718fab3c4a3daf22d45954e984a4"
format_stamp: "Formatted at 2026-08-07 15:15:44 on dist-test-slave-42z9"
server_key: "62fbc5b2425e2021752720d09640be96"
server_key_iv: "f59b6703adcf3ce6da67d029b29dbb4e"
server_key_version: "encryptionkey@0"
I20260807 15:15:44.969421 28940 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786115480851507-30472-0/minicluster-data/master-0/wal/instance:
uuid: "b96d718fab3c4a3daf22d45954e984a4"
format_stamp: "Formatted at 2026-08-07 15:15:44 on dist-test-slave-42z9"
server_key: "62fbc5b2425e2021752720d09640be96"
server_key_iv: "f59b6703adcf3ce6da67d029b29dbb4e"
server_key_version: "encryptionkey@0"
I20260807 15:15:44.970752 28940 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:15:44.971570 28963 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:15:44.971786 28940 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:15:44.971877 28940 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786115480851507-30472-0/minicluster-data/master-0/data,/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786115480851507-30472-0/minicluster-data/master-0/wal
uuid: "b96d718fab3c4a3daf22d45954e984a4"
format_stamp: "Formatted at 2026-08-07 15:15:44 on dist-test-slave-42z9"
server_key: "62fbc5b2425e2021752720d09640be96"
server_key_iv: "f59b6703adcf3ce6da67d029b29dbb4e"
server_key_version: "encryptionkey@0"
I20260807 15:15:44.971943 28940 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786115480851507-30472-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786115480851507-30472-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786115480851507-30472-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:15:44.975394 28940 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:15:44.977295 28940 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:15:44.977430 28940 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:15:44.981392 28940 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43291
I20260807 15:15:44.981429 29039 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43291 every 8 connection(s)
I20260807 15:15:44.981782 28940 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786115480851507-30472-0/minicluster-data/master-0/data/info.pb
I20260807 15:15:44.982769 29041 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:15:44.985052 29041 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b96d718fab3c4a3daf22d45954e984a4: Bootstrap starting.
I20260807 15:15:44.985678 29041 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b96d718fab3c4a3daf22d45954e984a4: Neither blocks nor log segments found. Creating new log.
I20260807 15:15:44.985951 29041 log.cc:824] T 00000000000000000000000000000000 P b96d718fab3c4a3daf22d45954e984a4: Log is configured to *not* fsync() on all Append() calls
I20260807 15:15:44.986673 29041 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b96d718fab3c4a3daf22d45954e984a4: No bootstrap required, opened a new log
I20260807 15:15:44.988294 29041 raft_consensus.cc:348] T 00000000000000000000000000000000 P b96d718fab3c4a3daf22d45954e984a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b96d718fab3c4a3daf22d45954e984a4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43291 } }
I20260807 15:15:44.988442 29041 raft_consensus.cc:374] T 00000000000000000000000000000000 P b96d718fab3c4a3daf22d45954e984a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:15:44.988466 29041 raft_consensus.cc:729] T 00000000000000000000000000000000 P b96d718fab3c4a3daf22d45954e984a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b96d718fab3c4a3daf22d45954e984a4, State: Initialized, Role: FOLLOWER
I20260807 15:15:44.988597 29041 consensus_queue.cc:260] T 00000000000000000000000000000000 P b96d718fab3c4a3daf22d45954e984a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b96d718fab3c4a3daf22d45954e984a4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43291 } }
I20260807 15:15:44.988665 29041 raft_consensus.cc:388] T 00000000000000000000000000000000 P b96d718fab3c4a3daf22d45954e984a4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:15:44.988686 29041 raft_consensus.cc:482] T 00000000000000000000000000000000 P b96d718fab3c4a3daf22d45954e984a4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:15:44.988716 29041 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b96d718fab3c4a3daf22d45954e984a4 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:15:44.989285 29041 raft_consensus.cc:504] T 00000000000000000000000000000000 P b96d718fab3c4a3daf22d45954e984a4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b96d718fab3c4a3daf22d45954e984a4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43291 } }
I20260807 15:15:44.989382 29041 leader_election.cc:302] T 00000000000000000000000000000000 P b96d718fab3c4a3daf22d45954e984a4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b96d718fab3c4a3daf22d45954e984a4; no voters:
I20260807 15:15:44.989596 29041 leader_election.cc:288] T 00000000000000000000000000000000 P b96d718fab3c4a3daf22d45954e984a4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 15:15:44.989679 29046 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b96d718fab3c4a3daf22d45954e984a4 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:15:44.989858 29046 raft_consensus.cc:686] T 00000000000000000000000000000000 P b96d718fab3c4a3daf22d45954e984a4 [term 1 LEADER]: Becoming Leader. State: Replica: b96d718fab3c4a3daf22d45954e984a4, State: Running, Role: LEADER
I20260807 15:15:44.989869 29041 sys_catalog.cc:564] T 00000000000000000000000000000000 P b96d718fab3c4a3daf22d45954e984a4 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:15:44.990031 29046 consensus_queue.cc:237] T 00000000000000000000000000000000 P b96d718fab3c4a3daf22d45954e984a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b96d718fab3c4a3daf22d45954e984a4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43291 } }
I20260807 15:15:44.990399 29047 sys_catalog.cc:455] T 00000000000000000000000000000000 P b96d718fab3c4a3daf22d45954e984a4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b96d718fab3c4a3daf22d45954e984a4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b96d718fab3c4a3daf22d45954e984a4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43291 } } }
I20260807 15:15:44.990552 29047 sys_catalog.cc:458] T 00000000000000000000000000000000 P b96d718fab3c4a3daf22d45954e984a4 [sys.catalog]: This master's current role is: LEADER
I20260807 15:15:44.990427 29051 sys_catalog.cc:455] T 00000000000000000000000000000000 P b96d718fab3c4a3daf22d45954e984a4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b96d718fab3c4a3daf22d45954e984a4. Latest consensus state: current_term: 1 leader_uuid: "b96d718fab3c4a3daf22d45954e984a4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b96d718fab3c4a3daf22d45954e984a4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43291 } } }
I20260807 15:15:44.990765 29054 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:15:44.991371 29054 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:15:44.991575 30472 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7fZSIL/build/release/bin/kudu as pid 28940
I20260807 15:15:44.991686 30472 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786115480851507-30472-0/minicluster-data/master-0/wal/instance
I20260807 15:15:44.991871 30472 external_mini_cluster.cc:1428] Setting key 48d1ef9868740a0b5f0d0afabc6a94bc
I20260807 15:15:44.990765 29051 sys_catalog.cc:458] T 00000000000000000000000000000000 P b96d718fab3c4a3daf22d45954e984a4 [sys.catalog]: This master's current role is: LEADER
I20260807 15:15:44.993029 29054 catalog_manager.cc:1347] Generated new cluster ID: b786165356b345a58b0aa209d5d4c833
I20260807 15:15:44.993083 29054 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:15:44.995400 29080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:44.992667 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41823 (local address 127.0.0.1:43291)
0807 15:15:44.992837 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:44.992846 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:44.993214 (+ 368us) server_negotiation.cc:407] Connection header received
0807 15:15:44.993346 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:44.993354 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:44.993394 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:44.993580 (+ 186us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:15:44.994798 (+ 1218us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:15:44.994802 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:15:44.994802 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:15:44.994806 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:15:44.994852 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:15:44.994868 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:15:44.994924 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0807 15:15:44.995145 (+ 221us) server_negotiation.cc:299] Negotiation successful
0807 15:15:44.995190 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":113,"thread_start_us":90,"threads_started":1}
I20260807 15:15:44.996857 30472 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 15:15:45.003046 29080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:45.001805 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52165 (local address 127.0.0.1:43291)
0807 15:15:45.001862 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:45.001864 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:45.002029 (+ 165us) server_negotiation.cc:407] Connection header received
0807 15:15:45.002087 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:45.002088 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:45.002094 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:45.002148 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:15:45.002818 (+ 670us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:15:45.002820 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:15:45.002820 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:15:45.002825 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:15:45.002843 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:15:45.002848 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:15:45.002910 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0807 15:15:45.002953 (+ 43us) server_negotiation.cc:299] Negotiation successful
0807 15:15:45.002968 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260807 15:15:45.010433 29054 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:15:45.010609 29054 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:15:45.019845 29054 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b96d718fab3c4a3daf22d45954e984a4: Generated new TSK 0
I20260807 15:15:45.020066 29054 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 15:15:45.029083 30472 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7fZSIL/build/release/bin/kudu with pid 28940
2026-08-07T15:15:45Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1435 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786115480851507-30472-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 15:15:45 dist-test-slave-42z9 krb5kdc[29093](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 15:15:45 dist-test-slave-42z9 krb5kdc[29093](info): set up 2 sockets
Aug 07 15:15:45 dist-test-slave-42z9 krb5kdc[29093](info): commencing operation
krb5kdc: starting...
W20260807 15:15:46.184896 30472 mini_kdc.cc:121] Time spent starting KDC: real 1.133s user 0.003s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786115480851507-30472-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786115480851507-30472-0/krb5kdc/test-user.keytab.
Aug 07 15:15:46 dist-test-slave-42z9 krb5kdc[29093](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115746, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T15:15:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T15:15:46Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786115480851507-30472-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786115480851507-30472-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 15:15:46.312929 30472 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7fZSIL/build/release/bin/kudu
/tmp/dist-test-task7fZSIL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786115480851507-30472-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786115480851507-30472-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786115480851507-30472-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786115480851507-30472-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40613
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786115480851507-30472-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38939
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40613
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786115480851507-30472-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786115480851507-30472-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786115480851507-30472-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:15:46.397291 29115 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:15:46.397563 29115 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:15:46.397599 29115 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:15:46.398801 29115 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:15:46.398849 29115 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:15:46.398869 29115 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:15:46.398885 29115 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:15:46.398900 29115 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:15:46.400494 29115 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38939
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786115480851507-30472-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786115480851507-30472-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40613
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786115480851507-30472-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40613
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786115480851507-30472-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29115
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786115480851507-30472-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:15:46.400759 29115 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:15:46.400995 29115 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:15:46.406059 29123 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:15:46.406090 29127 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:15:46.406136 29124 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:15:46.406172 29115 server_base.cc:1034] running on GCE node
I20260807 15:15:46.406585 29115 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:15:46.406893 29115 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:15:46.408049 29115 hybrid_clock.cc:648] HybridClock initialized: now 1786115746408024 us; error 41 us; skew 500 ppm
Aug 07 15:15:46 dist-test-slave-42z9 krb5kdc[29093](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115746, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:15:46.410583 29115 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 15:15:46.411046 29115 webserver.cc:460] Webserver started at http://127.0.0.1:35925/ using document root <none> and password file <none>
I20260807 15:15:46.411274 29115 fs_manager.cc:359] Metadata directory not provided
I20260807 15:15:46.411321 29115 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:15:46.411430 29115 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:15:46.412485 29115 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786115480851507-30472-0/minicluster-data/master-0/data/instance:
uuid: "6f8d94d3355f44dba71451829c98bade"
format_stamp: "Formatted at 2026-08-07 15:15:46 on dist-test-slave-42z9"
server_key: "011c08f20a7dd344af6e886a0501d75c"
server_key_iv: "c06dfd8ddaa4601db9ec6827d732f01c"
server_key_version: "encryptionkey@0"
I20260807 15:15:46.412837 29115 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786115480851507-30472-0/minicluster-data/master-0/wal/instance:
uuid: "6f8d94d3355f44dba71451829c98bade"
format_stamp: "Formatted at 2026-08-07 15:15:46 on dist-test-slave-42z9"
server_key: "011c08f20a7dd344af6e886a0501d75c"
server_key_iv: "c06dfd8ddaa4601db9ec6827d732f01c"
server_key_version: "encryptionkey@0"
I20260807 15:15:46.414243 29115 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260807 15:15:46.415110 29139 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:15:46.415288 29115 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260807 15:15:46.415524 29115 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786115480851507-30472-0/minicluster-data/master-0/data,/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786115480851507-30472-0/minicluster-data/master-0/wal
uuid: "6f8d94d3355f44dba71451829c98bade"
format_stamp: "Formatted at 2026-08-07 15:15:46 on dist-test-slave-42z9"
server_key: "011c08f20a7dd344af6e886a0501d75c"
server_key_iv: "c06dfd8ddaa4601db9ec6827d732f01c"
server_key_version: "encryptionkey@0"
I20260807 15:15:46.415758 29115 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786115480851507-30472-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786115480851507-30472-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786115480851507-30472-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:15:46.442085 29115 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:15:46.443953 29115 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:15:46.444077 29115 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:15:46.448062 29115 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40613
I20260807 15:15:46.448108 29214 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40613 every 8 connection(s)
I20260807 15:15:46.448423 29115 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786115480851507-30472-0/minicluster-data/master-0/data/info.pb
I20260807 15:15:46.449049 30472 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7fZSIL/build/release/bin/kudu as pid 29115
I20260807 15:15:46.449177 30472 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786115480851507-30472-0/minicluster-data/master-0/wal/instance
I20260807 15:15:46.449373 30472 external_mini_cluster.cc:1428] Setting key 2b3622d82057f96e8544a2402f2bfd76
I20260807 15:15:46.449513 29215 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:15:46.452735 29215 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6f8d94d3355f44dba71451829c98bade: Bootstrap starting.
I20260807 15:15:46.453497 29215 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6f8d94d3355f44dba71451829c98bade: Neither blocks nor log segments found. Creating new log.
I20260807 15:15:46.453787 29215 log.cc:824] T 00000000000000000000000000000000 P 6f8d94d3355f44dba71451829c98bade: Log is configured to *not* fsync() on all Append() calls
I20260807 15:15:46.454505 29215 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6f8d94d3355f44dba71451829c98bade: No bootstrap required, opened a new log
Aug 07 15:15:46 dist-test-slave-42z9 krb5kdc[29093](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115746, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260807 15:15:46.456707 29215 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6f8d94d3355f44dba71451829c98bade [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f8d94d3355f44dba71451829c98bade" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40613 } }
I20260807 15:15:46.456892 29215 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6f8d94d3355f44dba71451829c98bade [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:15:46.456924 29215 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6f8d94d3355f44dba71451829c98bade [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8d94d3355f44dba71451829c98bade, State: Initialized, Role: FOLLOWER
I20260807 15:15:46.457051 29215 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6f8d94d3355f44dba71451829c98bade [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f8d94d3355f44dba71451829c98bade" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40613 } }
I20260807 15:15:46.457132 29215 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6f8d94d3355f44dba71451829c98bade [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:15:46.457163 29215 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6f8d94d3355f44dba71451829c98bade [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:15:46.457216 29215 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6f8d94d3355f44dba71451829c98bade [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:15:46.458015 29215 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6f8d94d3355f44dba71451829c98bade [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f8d94d3355f44dba71451829c98bade" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40613 } }
I20260807 15:15:46.458155 29215 leader_election.cc:302] T 00000000000000000000000000000000 P 6f8d94d3355f44dba71451829c98bade [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6f8d94d3355f44dba71451829c98bade; no voters:
I20260807 15:15:46.458319 29215 leader_election.cc:288] T 00000000000000000000000000000000 P 6f8d94d3355f44dba71451829c98bade [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 15:15:46.458360 29221 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6f8d94d3355f44dba71451829c98bade [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:15:46.458516 29221 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6f8d94d3355f44dba71451829c98bade [term 1 LEADER]: Becoming Leader. State: Replica: 6f8d94d3355f44dba71451829c98bade, State: Running, Role: LEADER
I20260807 15:15:46.458627 29215 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6f8d94d3355f44dba71451829c98bade [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:15:46.458644 29221 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6f8d94d3355f44dba71451829c98bade [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f8d94d3355f44dba71451829c98bade" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40613 } }
I20260807 15:15:46.459080 29224 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6f8d94d3355f44dba71451829c98bade [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6f8d94d3355f44dba71451829c98bade. Latest consensus state: current_term: 1 leader_uuid: "6f8d94d3355f44dba71451829c98bade" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f8d94d3355f44dba71451829c98bade" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40613 } } }
I20260807 15:15:46.459160 29224 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6f8d94d3355f44dba71451829c98bade [sys.catalog]: This master's current role is: LEADER
I20260807 15:15:46.459321 29222 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6f8d94d3355f44dba71451829c98bade [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6f8d94d3355f44dba71451829c98bade" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f8d94d3355f44dba71451829c98bade" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40613 } } }
I20260807 15:15:46.459400 29222 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6f8d94d3355f44dba71451829c98bade [sys.catalog]: This master's current role is: LEADER
I20260807 15:15:46.459787 29230 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:15:46.460404 29230 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:15:46.461071 29218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:46.449916 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.33:60219 (local address 127.0.0.1:40613)
0807 15:15:46.450089 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:46.450099 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:46.450127 (+ 28us) server_negotiation.cc:407] Connection header received
0807 15:15:46.450639 (+ 512us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:46.450646 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:46.450695 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:46.450810 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:15:46.451594 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:46.452561 (+ 967us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:46.453577 (+ 1016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:46.454021 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:46.454105 (+ 84us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:46.456585 (+ 2480us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:15:46.456590 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:15:46.456590 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:15:46.456604 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:15:46.458658 (+ 2054us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:15:46.459156 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:15:46.459160 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:15:46.459160 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:15:46.459278 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:15:46.459573 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:15:46.459574 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:15:46.459574 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:15:46.459911 (+ 337us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:15:46.460498 (+ 587us) server_negotiation.cc:1035] Waiting for connection context
0807 15:15:46.460758 (+ 260us) server_negotiation.cc:299] Negotiation successful
0807 15:15:46.460868 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119,"thread_start_us":73,"threads_started":1}
I20260807 15:15:46.462045 29230 catalog_manager.cc:1347] Generated new cluster ID: 07add5f8da5f45d7a82654e7ab43019e
I20260807 15:15:46.462177 29230 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:15:46.462170 30472 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 15:15:46.471156 29218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:46.463714 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.33:54889 (local address 127.0.0.1:40613)
0807 15:15:46.463753 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:46.463755 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:46.463808 (+ 53us) server_negotiation.cc:407] Connection header received
0807 15:15:46.463884 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:46.463885 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:46.463894 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:46.463935 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:15:46.464747 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:46.465522 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:46.466516 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:46.466842 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:46.466905 (+ 63us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:46.468350 (+ 1445us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:15:46.468351 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:15:46.468352 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:15:46.468359 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:15:46.469946 (+ 1587us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:15:46.470388 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:15:46.470389 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:15:46.470389 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:15:46.470445 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:15:46.470658 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:15:46.470659 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:15:46.470659 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:15:46.470775 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:15:46.470862 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0807 15:15:46.470995 (+ 133us) server_negotiation.cc:299] Negotiation successful
0807 15:15:46.471061 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260807 15:15:46.501636 29230 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:15:46.502188 29230 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:15:46.515128 29230 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6f8d94d3355f44dba71451829c98bade: Generated new TSK 0
I20260807 15:15:46.515339 29230 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 15:15:46.518668 30472 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7fZSIL/build/release/bin/kudu with pid 29115
W20260807 15:15:46.535472 29091 connection.cc:535] client connection to 127.0.0.1:40613 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:40613): Connection reset by peer (error 104)
2026-08-07T15:15:46Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1497 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786115480851507-30472-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 15:15:46 dist-test-slave-42z9 krb5kdc[29264](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 15:15:46 dist-test-slave-42z9 krb5kdc[29264](info): set up 2 sockets
Aug 07 15:15:46 dist-test-slave-42z9 krb5kdc[29264](info): commencing operation
krb5kdc: starting...
W20260807 15:15:47.677691 30472 mini_kdc.cc:121] Time spent starting KDC: real 1.129s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786115480851507-30472-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786115480851507-30472-0/krb5kdc/test-user.keytab.
Aug 07 15:15:47 dist-test-slave-42z9 krb5kdc[29264](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115747, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T15:15:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T15:15:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786115480851507-30472-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786115480851507-30472-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 15:15:47.814570 30472 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7fZSIL/build/release/bin/kudu
/tmp/dist-test-task7fZSIL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786115480851507-30472-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786115480851507-30472-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786115480851507-30472-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786115480851507-30472-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45313
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786115480851507-30472-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42407
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45313
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786115480851507-30472-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786115480851507-30472-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786115480851507-30472-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:15:47.901726 29283 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:15:47.901973 29283 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:15:47.902012 29283 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:15:47.903571 29283 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:15:47.903650 29283 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:15:47.903676 29283 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:15:47.903695 29283 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:15:47.903712 29283 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:15:47.905512 29283 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42407
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786115480851507-30472-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786115480851507-30472-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45313
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786115480851507-30472-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45313
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786115480851507-30472-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29283
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786115480851507-30472-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:15:47.905810 29283 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:15:47.906050 29283 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:15:47.911644 29292 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:15:47.911706 29296 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:15:47.911695 29289 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:15:47.911736 29283 server_base.cc:1034] running on GCE node
I20260807 15:15:47.912096 29283 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:15:47.912367 29283 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:15:47.913506 29283 hybrid_clock.cc:648] HybridClock initialized: now 1786115747913483 us; error 35 us; skew 500 ppm
Aug 07 15:15:47 dist-test-slave-42z9 krb5kdc[29264](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115747, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:15:47.916102 29283 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 15:15:47.916554 29283 webserver.cc:460] Webserver started at http://127.0.0.1:37251/ using document root <none> and password file <none>
I20260807 15:15:47.916785 29283 fs_manager.cc:359] Metadata directory not provided
I20260807 15:15:47.916833 29283 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:15:47.916973 29283 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:15:47.918080 29283 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786115480851507-30472-0/minicluster-data/master-0/data/instance:
uuid: "9b0279e72641428ba4c1370c742cbae1"
format_stamp: "Formatted at 2026-08-07 15:15:47 on dist-test-slave-42z9"
server_key: "3a8685a679364847d28632ec8e1f7b91"
server_key_iv: "dbfae39c053e0c51aa4d73a84452a090"
server_key_version: "encryptionkey@0"
I20260807 15:15:47.918416 29283 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786115480851507-30472-0/minicluster-data/master-0/wal/instance:
uuid: "9b0279e72641428ba4c1370c742cbae1"
format_stamp: "Formatted at 2026-08-07 15:15:47 on dist-test-slave-42z9"
server_key: "3a8685a679364847d28632ec8e1f7b91"
server_key_iv: "dbfae39c053e0c51aa4d73a84452a090"
server_key_version: "encryptionkey@0"
I20260807 15:15:47.919800 29283 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 15:15:47.920704 29305 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:15:47.920969 29283 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:15:47.921068 29283 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786115480851507-30472-0/minicluster-data/master-0/data,/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786115480851507-30472-0/minicluster-data/master-0/wal
uuid: "9b0279e72641428ba4c1370c742cbae1"
format_stamp: "Formatted at 2026-08-07 15:15:47 on dist-test-slave-42z9"
server_key: "3a8685a679364847d28632ec8e1f7b91"
server_key_iv: "dbfae39c053e0c51aa4d73a84452a090"
server_key_version: "encryptionkey@0"
I20260807 15:15:47.921139 29283 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786115480851507-30472-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786115480851507-30472-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786115480851507-30472-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:15:47.944208 29283 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:15:47.947540 29283 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:15:47.947712 29283 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:15:47.953146 29283 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45313
I20260807 15:15:47.953220 29378 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45313 every 8 connection(s)
I20260807 15:15:47.953642 29283 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786115480851507-30472-0/minicluster-data/master-0/data/info.pb
I20260807 15:15:47.954607 29379 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:15:47.956991 29379 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9b0279e72641428ba4c1370c742cbae1: Bootstrap starting.
I20260807 15:15:47.957675 29379 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9b0279e72641428ba4c1370c742cbae1: Neither blocks nor log segments found. Creating new log.
I20260807 15:15:47.957978 29379 log.cc:824] T 00000000000000000000000000000000 P 9b0279e72641428ba4c1370c742cbae1: Log is configured to *not* fsync() on all Append() calls
I20260807 15:15:47.958838 29379 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9b0279e72641428ba4c1370c742cbae1: No bootstrap required, opened a new log
I20260807 15:15:47.960574 29379 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9b0279e72641428ba4c1370c742cbae1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b0279e72641428ba4c1370c742cbae1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45313 } }
I20260807 15:15:47.960741 29379 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9b0279e72641428ba4c1370c742cbae1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:15:47.960763 29379 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9b0279e72641428ba4c1370c742cbae1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9b0279e72641428ba4c1370c742cbae1, State: Initialized, Role: FOLLOWER
I20260807 15:15:47.960902 29379 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9b0279e72641428ba4c1370c742cbae1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b0279e72641428ba4c1370c742cbae1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45313 } }
I20260807 15:15:47.960974 29379 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9b0279e72641428ba4c1370c742cbae1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:15:47.960997 29379 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9b0279e72641428ba4c1370c742cbae1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:15:47.961051 29379 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9b0279e72641428ba4c1370c742cbae1 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:15:47.961709 29379 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9b0279e72641428ba4c1370c742cbae1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b0279e72641428ba4c1370c742cbae1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45313 } }
I20260807 15:15:47.961807 29379 leader_election.cc:302] T 00000000000000000000000000000000 P 9b0279e72641428ba4c1370c742cbae1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9b0279e72641428ba4c1370c742cbae1; no voters:
I20260807 15:15:47.962006 29379 leader_election.cc:288] T 00000000000000000000000000000000 P 9b0279e72641428ba4c1370c742cbae1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 15:15:47.962182 29384 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9b0279e72641428ba4c1370c742cbae1 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:15:47.962318 29379 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9b0279e72641428ba4c1370c742cbae1 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:15:47.962383 30472 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7fZSIL/build/release/bin/kudu as pid 29283
I20260807 15:15:47.962497 30472 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786115480851507-30472-0/minicluster-data/master-0/wal/instance
I20260807 15:15:47.962504 29384 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9b0279e72641428ba4c1370c742cbae1 [term 1 LEADER]: Becoming Leader. State: Replica: 9b0279e72641428ba4c1370c742cbae1, State: Running, Role: LEADER
I20260807 15:15:47.962631 29384 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9b0279e72641428ba4c1370c742cbae1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b0279e72641428ba4c1370c742cbae1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45313 } }
I20260807 15:15:47.962728 30472 external_mini_cluster.cc:1428] Setting key 10acaf8c531c626df8ac18c6a43551bb
I20260807 15:15:47.963420 29386 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9b0279e72641428ba4c1370c742cbae1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9b0279e72641428ba4c1370c742cbae1. Latest consensus state: current_term: 1 leader_uuid: "9b0279e72641428ba4c1370c742cbae1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b0279e72641428ba4c1370c742cbae1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45313 } } }
I20260807 15:15:47.963549 29386 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9b0279e72641428ba4c1370c742cbae1 [sys.catalog]: This master's current role is: LEADER
I20260807 15:15:47.964066 29385 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9b0279e72641428ba4c1370c742cbae1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9b0279e72641428ba4c1370c742cbae1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b0279e72641428ba4c1370c742cbae1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45313 } } }
I20260807 15:15:47.964315 29385 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9b0279e72641428ba4c1370c742cbae1 [sys.catalog]: This master's current role is: LEADER
I20260807 15:15:47.964330 29405 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:15:47.964985 29405 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:15:47.966682 29405 catalog_manager.cc:1347] Generated new cluster ID: 8cb4e5d9820f4e1ea0c2d6dcf233e188
I20260807 15:15:47.966753 29405 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:15:47.970285 29403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:47.963542 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.33:48759 (local address 127.0.0.1:45313)
0807 15:15:47.963769 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:47.963780 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:47.963821 (+ 41us) server_negotiation.cc:407] Connection header received
0807 15:15:47.963973 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:47.963982 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:47.964053 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:47.964786 (+ 733us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:15:47.964931 (+ 145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:47.966501 (+ 1570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:47.967827 (+ 1326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:47.968530 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:47.968696 (+ 166us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:47.969433 (+ 737us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:15:47.969478 (+ 45us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:15:47.969479 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:15:47.969727 (+ 248us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:15:47.969770 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:15:47.969788 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:15:47.969886 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0807 15:15:47.970019 (+ 133us) server_negotiation.cc:299] Negotiation successful
0807 15:15:47.970053 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":164,"thread_start_us":73,"threads_started":1}
I20260807 15:15:47.971395 30472 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 15:15:47.976770 29403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:47.972910 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.33:55319 (local address 127.0.0.1:45313)
0807 15:15:47.973024 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:47.973026 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:47.973159 (+ 133us) server_negotiation.cc:407] Connection header received
0807 15:15:47.973215 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:47.973215 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:47.973227 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:47.973283 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:15:47.973458 (+ 175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:47.974226 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:47.975291 (+ 1065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:47.975603 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:47.975657 (+ 54us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:47.976457 (+ 800us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:15:47.976458 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:15:47.976459 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:15:47.976463 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:15:47.976478 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:15:47.976483 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:15:47.976534 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0807 15:15:47.976644 (+ 110us) server_negotiation.cc:299] Negotiation successful
0807 15:15:47.976660 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260807 15:15:47.982007 29405 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:15:47.982525 29405 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:15:47.992648 29405 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9b0279e72641428ba4c1370c742cbae1: Generated new TSK 0
I20260807 15:15:47.992852 29405 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 15:15:47.997740 30472 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7fZSIL/build/release/bin/kudu with pid 29283
W20260807 15:15:48.014168 29260 connection.cc:535] client connection to 127.0.0.1:45313 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:45313): Connection reset by peer (error 104)
2026-08-07T15:15:48Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1479 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786115480851507-30472-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 15:15:48 dist-test-slave-42z9 krb5kdc[29425](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 15:15:48 dist-test-slave-42z9 krb5kdc[29425](info): set up 2 sockets
Aug 07 15:15:48 dist-test-slave-42z9 krb5kdc[29425](info): commencing operation
krb5kdc: starting...
W20260807 15:15:49.143740 30472 mini_kdc.cc:121] Time spent starting KDC: real 1.116s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786115480851507-30472-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786115480851507-30472-0/krb5kdc/test-user.keytab.
Aug 07 15:15:49 dist-test-slave-42z9 krb5kdc[29425](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115749, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T15:15:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T15:15:49Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786115480851507-30472-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786115480851507-30472-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 15:15:49.281687 30472 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7fZSIL/build/release/bin/kudu
/tmp/dist-test-task7fZSIL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786115480851507-30472-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786115480851507-30472-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786115480851507-30472-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786115480851507-30472-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40023
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786115480851507-30472-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35999
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40023
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786115480851507-30472-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786115480851507-30472-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786115480851507-30472-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:15:49.365629 29442 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:15:49.365861 29442 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:15:49.365895 29442 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:15:49.367136 29442 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:15:49.367190 29442 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:15:49.367208 29442 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:15:49.367225 29442 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:15:49.367240 29442 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:15:49.368814 29442 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35999
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786115480851507-30472-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786115480851507-30472-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40023
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786115480851507-30472-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40023
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786115480851507-30472-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29442
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786115480851507-30472-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:15:49.369077 29442 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:15:49.369300 29442 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:15:49.374351 29451 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:15:49.374351 29448 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:15:49.374353 29449 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:15:49.374378 29442 server_base.cc:1034] running on GCE node
I20260807 15:15:49.374820 29442 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:15:49.375113 29442 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:15:49.376281 29442 hybrid_clock.cc:648] HybridClock initialized: now 1786115749376267 us; error 30 us; skew 500 ppm
Aug 07 15:15:49 dist-test-slave-42z9 krb5kdc[29425](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115749, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:15:49.385421 29442 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 15:15:49.385973 29442 webserver.cc:460] Webserver started at http://127.0.0.1:37917/ using document root <none> and password file <none>
I20260807 15:15:49.386215 29442 fs_manager.cc:359] Metadata directory not provided
I20260807 15:15:49.386267 29442 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:15:49.386407 29442 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:15:49.387501 29442 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786115480851507-30472-0/minicluster-data/master-0/data/instance:
uuid: "ada3c44058c2495291c0c487e52e0e67"
format_stamp: "Formatted at 2026-08-07 15:15:49 on dist-test-slave-42z9"
server_key: "546e4ffde38e7cdd83c1dd1cbba6159e"
server_key_iv: "a632f2c5aadbd837a9a3239ba9ff2f76"
server_key_version: "encryptionkey@0"
I20260807 15:15:49.387842 29442 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786115480851507-30472-0/minicluster-data/master-0/wal/instance:
uuid: "ada3c44058c2495291c0c487e52e0e67"
format_stamp: "Formatted at 2026-08-07 15:15:49 on dist-test-slave-42z9"
server_key: "546e4ffde38e7cdd83c1dd1cbba6159e"
server_key_iv: "a632f2c5aadbd837a9a3239ba9ff2f76"
server_key_version: "encryptionkey@0"
I20260807 15:15:49.389214 29442 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260807 15:15:49.390360 29459 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:15:49.390667 29442 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:15:49.390780 29442 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786115480851507-30472-0/minicluster-data/master-0/data,/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786115480851507-30472-0/minicluster-data/master-0/wal
uuid: "ada3c44058c2495291c0c487e52e0e67"
format_stamp: "Formatted at 2026-08-07 15:15:49 on dist-test-slave-42z9"
server_key: "546e4ffde38e7cdd83c1dd1cbba6159e"
server_key_iv: "a632f2c5aadbd837a9a3239ba9ff2f76"
server_key_version: "encryptionkey@0"
I20260807 15:15:49.390863 29442 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786115480851507-30472-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786115480851507-30472-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786115480851507-30472-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:15:49.394454 29442 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:15:49.396695 29442 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:15:49.396842 29442 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:15:49.401219 29442 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40023
I20260807 15:15:49.401255 29545 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40023 every 8 connection(s)
I20260807 15:15:49.401643 29442 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786115480851507-30472-0/minicluster-data/master-0/data/info.pb
I20260807 15:15:49.403075 29546 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:15:49.406605 29546 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ada3c44058c2495291c0c487e52e0e67: Bootstrap starting.
I20260807 15:15:49.407275 29546 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ada3c44058c2495291c0c487e52e0e67: Neither blocks nor log segments found. Creating new log.
I20260807 15:15:49.407584 29546 log.cc:824] T 00000000000000000000000000000000 P ada3c44058c2495291c0c487e52e0e67: Log is configured to *not* fsync() on all Append() calls
I20260807 15:15:49.408633 29546 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ada3c44058c2495291c0c487e52e0e67: No bootstrap required, opened a new log
I20260807 15:15:49.408761 30472 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7fZSIL/build/release/bin/kudu as pid 29442
I20260807 15:15:49.408850 30472 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786115480851507-30472-0/minicluster-data/master-0/wal/instance
I20260807 15:15:49.409045 30472 external_mini_cluster.cc:1428] Setting key 7e4465d7c9a456f7a9ebf736918c3fb4
I20260807 15:15:49.411686 29546 raft_consensus.cc:348] T 00000000000000000000000000000000 P ada3c44058c2495291c0c487e52e0e67 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ada3c44058c2495291c0c487e52e0e67" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40023 } }
I20260807 15:15:49.411876 29546 raft_consensus.cc:374] T 00000000000000000000000000000000 P ada3c44058c2495291c0c487e52e0e67 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:15:49.411966 29546 raft_consensus.cc:729] T 00000000000000000000000000000000 P ada3c44058c2495291c0c487e52e0e67 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ada3c44058c2495291c0c487e52e0e67, State: Initialized, Role: FOLLOWER
I20260807 15:15:49.412118 29546 consensus_queue.cc:260] T 00000000000000000000000000000000 P ada3c44058c2495291c0c487e52e0e67 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ada3c44058c2495291c0c487e52e0e67" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40023 } }
I20260807 15:15:49.412215 29546 raft_consensus.cc:388] T 00000000000000000000000000000000 P ada3c44058c2495291c0c487e52e0e67 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:15:49.412248 29546 raft_consensus.cc:482] T 00000000000000000000000000000000 P ada3c44058c2495291c0c487e52e0e67 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:15:49.412298 29546 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ada3c44058c2495291c0c487e52e0e67 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:15:49.412300 29551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:49.409597 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.33:55421 (local address 127.0.0.1:40023)
0807 15:15:49.409771 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:49.409779 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:49.409805 (+ 26us) server_negotiation.cc:407] Connection header received
0807 15:15:49.410269 (+ 464us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:49.410277 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:49.410303 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:49.410426 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:15:49.411431 (+ 1005us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:15:49.411436 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:15:49.411437 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:15:49.411443 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:15:49.411505 (+ 62us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:15:49.411520 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:15:49.411617 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0807 15:15:49.411725 (+ 108us) server_negotiation.cc:299] Negotiation successful
0807 15:15:49.411783 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":72,"threads_started":1}
I20260807 15:15:49.413138 29546 raft_consensus.cc:504] T 00000000000000000000000000000000 P ada3c44058c2495291c0c487e52e0e67 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ada3c44058c2495291c0c487e52e0e67" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40023 } }
I20260807 15:15:49.413245 29546 leader_election.cc:302] T 00000000000000000000000000000000 P ada3c44058c2495291c0c487e52e0e67 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ada3c44058c2495291c0c487e52e0e67; no voters:
I20260807 15:15:49.413622 29546 leader_election.cc:288] T 00000000000000000000000000000000 P ada3c44058c2495291c0c487e52e0e67 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 15:15:49.413674 29553 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ada3c44058c2495291c0c487e52e0e67 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:15:49.413913 30472 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 15:15:49.413980 29553 raft_consensus.cc:686] T 00000000000000000000000000000000 P ada3c44058c2495291c0c487e52e0e67 [term 1 LEADER]: Becoming Leader. State: Replica: ada3c44058c2495291c0c487e52e0e67, State: Running, Role: LEADER
I20260807 15:15:49.414089 29553 consensus_queue.cc:237] T 00000000000000000000000000000000 P ada3c44058c2495291c0c487e52e0e67 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ada3c44058c2495291c0c487e52e0e67" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40023 } }
I20260807 15:15:49.414109 29546 sys_catalog.cc:564] T 00000000000000000000000000000000 P ada3c44058c2495291c0c487e52e0e67 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:15:49.414587 29556 sys_catalog.cc:455] T 00000000000000000000000000000000 P ada3c44058c2495291c0c487e52e0e67 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ada3c44058c2495291c0c487e52e0e67. Latest consensus state: current_term: 1 leader_uuid: "ada3c44058c2495291c0c487e52e0e67" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ada3c44058c2495291c0c487e52e0e67" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40023 } } }
I20260807 15:15:49.414680 29556 sys_catalog.cc:458] T 00000000000000000000000000000000 P ada3c44058c2495291c0c487e52e0e67 [sys.catalog]: This master's current role is: LEADER
I20260807 15:15:49.414901 29555 sys_catalog.cc:455] T 00000000000000000000000000000000 P ada3c44058c2495291c0c487e52e0e67 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ada3c44058c2495291c0c487e52e0e67" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ada3c44058c2495291c0c487e52e0e67" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40023 } } }
I20260807 15:15:49.414969 29555 sys_catalog.cc:458] T 00000000000000000000000000000000 P ada3c44058c2495291c0c487e52e0e67 [sys.catalog]: This master's current role is: LEADER
I20260807 15:15:49.415041 29566 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:15:49.415726 29566 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:15:49.423388 29551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:49.422116 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.33:60371 (local address 127.0.0.1:40023)
0807 15:15:49.422168 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:49.422171 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:49.422304 (+ 133us) server_negotiation.cc:407] Connection header received
0807 15:15:49.422362 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:49.422364 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:49.422370 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:49.422448 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:15:49.423156 (+ 708us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:15:49.423158 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:15:49.423158 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:15:49.423163 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:15:49.423179 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:15:49.423183 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:15:49.423229 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0807 15:15:49.423299 (+ 70us) server_negotiation.cc:299] Negotiation successful
0807 15:15:49.423314 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260807 15:15:49.423578 29566 catalog_manager.cc:1347] Generated new cluster ID: 0b473957342045c3adc458aa74016ec3
I20260807 15:15:49.423645 29566 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:15:49.429915 29566 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:15:49.430074 29566 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:15:49.434712 29566 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ada3c44058c2495291c0c487e52e0e67: Generated new TSK 0
I20260807 15:15:49.434888 29566 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 15:15:49.437223 30472 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7fZSIL/build/release/bin/kudu with pid 29442
2026-08-07T15:15:49Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1436 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786115480851507-30472-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 15:15:49 dist-test-slave-42z9 krb5kdc[29601](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 15:15:49 dist-test-slave-42z9 krb5kdc[29601](info): set up 2 sockets
Aug 07 15:15:49 dist-test-slave-42z9 krb5kdc[29601](info): commencing operation
krb5kdc: starting...
W20260807 15:15:50.581254 30472 mini_kdc.cc:121] Time spent starting KDC: real 1.116s user 0.001s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786115480851507-30472-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786115480851507-30472-0/krb5kdc/test-user.keytab.
Aug 07 15:15:50 dist-test-slave-42z9 krb5kdc[29601](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115750, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T15:15:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T15:15:50Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786115480851507-30472-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786115480851507-30472-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 15:15:50.717301 30472 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7fZSIL/build/release/bin/kudu
/tmp/dist-test-task7fZSIL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786115480851507-30472-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786115480851507-30472-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786115480851507-30472-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786115480851507-30472-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33553
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786115480851507-30472-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34253
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33553
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786115480851507-30472-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786115480851507-30472-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786115480851507-30472-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:15:50.801731 29621 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:15:50.801939 29621 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:15:50.801960 29621 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:15:50.803088 29621 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:15:50.803131 29621 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:15:50.803144 29621 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:15:50.803155 29621 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:15:50.803166 29621 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:15:50.804661 29621 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34253
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786115480851507-30472-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786115480851507-30472-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33553
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786115480851507-30472-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33553
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786115480851507-30472-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29621
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786115480851507-30472-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:15:50.804888 29621 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:15:50.805124 29621 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:15:50.810272 29632 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:15:50.810288 29628 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:15:50.810292 29629 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:15:50.810385 29621 server_base.cc:1034] running on GCE node
I20260807 15:15:50.810783 29621 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:15:50.811087 29621 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:15:50.812273 29621 hybrid_clock.cc:648] HybridClock initialized: now 1786115750812251 us; error 35 us; skew 500 ppm
Aug 07 15:15:50 dist-test-slave-42z9 krb5kdc[29601](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115750, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:15:50.814924 29621 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 15:15:50.815414 29621 webserver.cc:460] Webserver started at http://127.0.0.1:43235/ using document root <none> and password file <none>
I20260807 15:15:50.815651 29621 fs_manager.cc:359] Metadata directory not provided
I20260807 15:15:50.815703 29621 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:15:50.815830 29621 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:15:50.817030 29621 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786115480851507-30472-0/minicluster-data/master-0/data/instance:
uuid: "e1d3f2d3263a4b0187fbe55dbba4669b"
format_stamp: "Formatted at 2026-08-07 15:15:50 on dist-test-slave-42z9"
server_key: "e44f95c357234c09d38218cf28e7566d"
server_key_iv: "7252c60e7d867a8a4a0469cfc49fd283"
server_key_version: "encryptionkey@0"
I20260807 15:15:50.817422 29621 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786115480851507-30472-0/minicluster-data/master-0/wal/instance:
uuid: "e1d3f2d3263a4b0187fbe55dbba4669b"
format_stamp: "Formatted at 2026-08-07 15:15:50 on dist-test-slave-42z9"
server_key: "e44f95c357234c09d38218cf28e7566d"
server_key_iv: "7252c60e7d867a8a4a0469cfc49fd283"
server_key_version: "encryptionkey@0"
I20260807 15:15:50.818958 29621 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:15:50.819909 29640 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:15:50.820130 29621 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:15:50.820226 29621 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786115480851507-30472-0/minicluster-data/master-0/data,/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786115480851507-30472-0/minicluster-data/master-0/wal
uuid: "e1d3f2d3263a4b0187fbe55dbba4669b"
format_stamp: "Formatted at 2026-08-07 15:15:50 on dist-test-slave-42z9"
server_key: "e44f95c357234c09d38218cf28e7566d"
server_key_iv: "7252c60e7d867a8a4a0469cfc49fd283"
server_key_version: "encryptionkey@0"
I20260807 15:15:50.820297 29621 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786115480851507-30472-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786115480851507-30472-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786115480851507-30472-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:15:50.842000 29621 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:15:50.844302 29621 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:15:50.844461 29621 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:15:50.849529 29621 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33553
I20260807 15:15:50.849752 29716 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33553 every 8 connection(s)
I20260807 15:15:50.849977 29621 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786115480851507-30472-0/minicluster-data/master-0/data/info.pb
I20260807 15:15:50.850984 29718 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:15:50.853505 29718 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e1d3f2d3263a4b0187fbe55dbba4669b: Bootstrap starting.
I20260807 15:15:50.854107 30472 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7fZSIL/build/release/bin/kudu as pid 29621
I20260807 15:15:50.854233 30472 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786115480851507-30472-0/minicluster-data/master-0/wal/instance
I20260807 15:15:50.854233 29718 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e1d3f2d3263a4b0187fbe55dbba4669b: Neither blocks nor log segments found. Creating new log.
I20260807 15:15:50.854449 30472 external_mini_cluster.cc:1428] Setting key ce65bfe97d096623f9a832e502cd7c47
I20260807 15:15:50.854565 29718 log.cc:824] T 00000000000000000000000000000000 P e1d3f2d3263a4b0187fbe55dbba4669b: Log is configured to *not* fsync() on all Append() calls
I20260807 15:15:50.855379 29718 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e1d3f2d3263a4b0187fbe55dbba4669b: No bootstrap required, opened a new log
I20260807 15:15:50.857386 29718 raft_consensus.cc:348] T 00000000000000000000000000000000 P e1d3f2d3263a4b0187fbe55dbba4669b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e1d3f2d3263a4b0187fbe55dbba4669b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33553 } }
I20260807 15:15:50.857589 29718 raft_consensus.cc:374] T 00000000000000000000000000000000 P e1d3f2d3263a4b0187fbe55dbba4669b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:15:50.857641 29718 raft_consensus.cc:729] T 00000000000000000000000000000000 P e1d3f2d3263a4b0187fbe55dbba4669b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1d3f2d3263a4b0187fbe55dbba4669b, State: Initialized, Role: FOLLOWER
I20260807 15:15:50.857771 29718 consensus_queue.cc:260] T 00000000000000000000000000000000 P e1d3f2d3263a4b0187fbe55dbba4669b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e1d3f2d3263a4b0187fbe55dbba4669b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33553 } }
I20260807 15:15:50.857849 29718 raft_consensus.cc:388] T 00000000000000000000000000000000 P e1d3f2d3263a4b0187fbe55dbba4669b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:15:50.857887 29718 raft_consensus.cc:482] T 00000000000000000000000000000000 P e1d3f2d3263a4b0187fbe55dbba4669b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:15:50.857932 29718 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e1d3f2d3263a4b0187fbe55dbba4669b [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:15:50.858525 29718 raft_consensus.cc:504] T 00000000000000000000000000000000 P e1d3f2d3263a4b0187fbe55dbba4669b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e1d3f2d3263a4b0187fbe55dbba4669b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33553 } }
I20260807 15:15:50.858634 29718 leader_election.cc:302] T 00000000000000000000000000000000 P e1d3f2d3263a4b0187fbe55dbba4669b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e1d3f2d3263a4b0187fbe55dbba4669b; no voters:
I20260807 15:15:50.858819 29718 leader_election.cc:288] T 00000000000000000000000000000000 P e1d3f2d3263a4b0187fbe55dbba4669b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 15:15:50.858886 29727 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e1d3f2d3263a4b0187fbe55dbba4669b [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:15:50.859095 29727 raft_consensus.cc:686] T 00000000000000000000000000000000 P e1d3f2d3263a4b0187fbe55dbba4669b [term 1 LEADER]: Becoming Leader. State: Replica: e1d3f2d3263a4b0187fbe55dbba4669b, State: Running, Role: LEADER
I20260807 15:15:50.859184 29718 sys_catalog.cc:564] T 00000000000000000000000000000000 P e1d3f2d3263a4b0187fbe55dbba4669b [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:15:50.859236 29727 consensus_queue.cc:237] T 00000000000000000000000000000000 P e1d3f2d3263a4b0187fbe55dbba4669b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e1d3f2d3263a4b0187fbe55dbba4669b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33553 } }
I20260807 15:15:50.859658 29727 sys_catalog.cc:455] T 00000000000000000000000000000000 P e1d3f2d3263a4b0187fbe55dbba4669b [sys.catalog]: SysCatalogTable state changed. Reason: New leader e1d3f2d3263a4b0187fbe55dbba4669b. Latest consensus state: current_term: 1 leader_uuid: "e1d3f2d3263a4b0187fbe55dbba4669b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e1d3f2d3263a4b0187fbe55dbba4669b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33553 } } }
I20260807 15:15:50.859724 29730 sys_catalog.cc:455] T 00000000000000000000000000000000 P e1d3f2d3263a4b0187fbe55dbba4669b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e1d3f2d3263a4b0187fbe55dbba4669b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e1d3f2d3263a4b0187fbe55dbba4669b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33553 } } }
I20260807 15:15:50.859797 29730 sys_catalog.cc:458] T 00000000000000000000000000000000 P e1d3f2d3263a4b0187fbe55dbba4669b [sys.catalog]: This master's current role is: LEADER
I20260807 15:15:50.860014 29727 sys_catalog.cc:458] T 00000000000000000000000000000000 P e1d3f2d3263a4b0187fbe55dbba4669b [sys.catalog]: This master's current role is: LEADER
I20260807 15:15:50.860173 29740 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:15:50.860723 29740 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 07 15:15:50 dist-test-slave-42z9 krb5kdc[29601](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115750, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260807 15:15:50.862663 29740 catalog_manager.cc:1347] Generated new cluster ID: efc244f9c6794afe8df7075e2feccd15
I20260807 15:15:50.862720 29740 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:15:50.866484 29723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:50.854953 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.33:50599 (local address 127.0.0.1:33553)
0807 15:15:50.855122 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:50.855130 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:50.855169 (+ 39us) server_negotiation.cc:407] Connection header received
0807 15:15:50.855704 (+ 535us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:50.855712 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:50.855764 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:50.855904 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:15:50.856551 (+ 647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:50.857504 (+ 953us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:50.858324 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:50.858779 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:50.858900 (+ 121us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:50.862026 (+ 3126us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:15:50.862031 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:15:50.862031 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:15:50.862045 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:15:50.864285 (+ 2240us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:15:50.864775 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:15:50.864778 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:15:50.864779 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:15:50.864915 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:15:50.865149 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:15:50.865150 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:15:50.865150 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:15:50.865521 (+ 371us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:15:50.865698 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0807 15:15:50.866009 (+ 311us) server_negotiation.cc:299] Negotiation successful
0807 15:15:50.866242 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":120,"thread_start_us":67,"threads_started":1}
I20260807 15:15:50.867434 30472 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 15:15:50.874719 29740 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:15:50.875396 29740 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:15:50.877661 29723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:50.869744 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.33:41781 (local address 127.0.0.1:33553)
0807 15:15:50.869789 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:50.869791 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:50.869966 (+ 175us) server_negotiation.cc:407] Connection header received
0807 15:15:50.870031 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:50.870031 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:50.870043 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:50.870095 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:15:50.870794 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:50.871594 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:50.872672 (+ 1078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:50.873005 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:50.873066 (+ 61us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:50.874674 (+ 1608us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:15:50.874676 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:15:50.874676 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:15:50.874684 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:15:50.876416 (+ 1732us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:15:50.876865 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:15:50.876865 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:15:50.876866 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:15:50.876923 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:15:50.877133 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:15:50.877134 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:15:50.877134 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:15:50.877250 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:15:50.877342 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0807 15:15:50.877506 (+ 164us) server_negotiation.cc:299] Negotiation successful
0807 15:15:50.877570 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260807 15:15:50.888027 29740 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e1d3f2d3263a4b0187fbe55dbba4669b: Generated new TSK 0
I20260807 15:15:50.888248 29740 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 15:15:50.897572 30472 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7fZSIL/build/release/bin/kudu with pid 29621
W20260807 15:15:50.911944 29597 connection.cc:535] client connection to 127.0.0.1:33553 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:33553): Connection reset by peer (error 104)
2026-08-07T15:15:50Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1460 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786115480851507-30472-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 15:15:50 dist-test-slave-42z9 krb5kdc[29768](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 15:15:50 dist-test-slave-42z9 krb5kdc[29768](info): set up 2 sockets
Aug 07 15:15:50 dist-test-slave-42z9 krb5kdc[29768](info): commencing operation
krb5kdc: starting...
W20260807 15:15:52.066617 30472 mini_kdc.cc:121] Time spent starting KDC: real 1.141s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786115480851507-30472-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786115480851507-30472-0/krb5kdc/test-user.keytab.
Aug 07 15:15:52 dist-test-slave-42z9 krb5kdc[29768](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115752, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T15:15:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T15:15:52Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786115480851507-30472-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786115480851507-30472-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 15:15:52.205965 30472 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7fZSIL/build/release/bin/kudu
/tmp/dist-test-task7fZSIL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786115480851507-30472-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786115480851507-30472-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786115480851507-30472-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786115480851507-30472-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45555
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786115480851507-30472-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36551
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45555
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786115480851507-30472-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786115480851507-30472-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786115480851507-30472-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:15:52.293718 29789 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:15:52.293956 29789 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:15:52.293982 29789 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:15:52.295220 29789 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:15:52.295272 29789 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:15:52.295285 29789 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:15:52.295296 29789 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:15:52.295307 29789 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:15:52.296917 29789 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36551
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786115480851507-30472-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786115480851507-30472-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45555
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786115480851507-30472-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45555
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786115480851507-30472-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29789
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786115480851507-30472-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:15:52.297173 29789 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:15:52.297412 29789 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:15:52.302531 29796 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:15:52.302727 29799 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:15:52.302698 29797 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:15:52.302904 29789 server_base.cc:1034] running on GCE node
I20260807 15:15:52.303182 29789 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:15:52.303511 29789 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:15:52.304682 29789 hybrid_clock.cc:648] HybridClock initialized: now 1786115752304670 us; error 33 us; skew 500 ppm
Aug 07 15:15:52 dist-test-slave-42z9 krb5kdc[29768](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115752, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:15:52.307204 29789 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 15:15:52.307705 29789 webserver.cc:460] Webserver started at http://127.0.0.1:42979/ using document root <none> and password file <none>
I20260807 15:15:52.307941 29789 fs_manager.cc:359] Metadata directory not provided
I20260807 15:15:52.307993 29789 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:15:52.308130 29789 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:15:52.309191 29789 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786115480851507-30472-0/minicluster-data/master-0/data/instance:
uuid: "6652f599c801480e98e8e7b3c5a95c21"
format_stamp: "Formatted at 2026-08-07 15:15:52 on dist-test-slave-42z9"
server_key: "3c597f827f82d4cb9e4d9bb35c8cadd9"
server_key_iv: "49af32a1027c5ead6035f074683f5618"
server_key_version: "encryptionkey@0"
I20260807 15:15:52.309578 29789 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786115480851507-30472-0/minicluster-data/master-0/wal/instance:
uuid: "6652f599c801480e98e8e7b3c5a95c21"
format_stamp: "Formatted at 2026-08-07 15:15:52 on dist-test-slave-42z9"
server_key: "3c597f827f82d4cb9e4d9bb35c8cadd9"
server_key_iv: "49af32a1027c5ead6035f074683f5618"
server_key_version: "encryptionkey@0"
I20260807 15:15:52.311087 29789 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260807 15:15:52.311928 29809 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:15:52.312266 29789 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 15:15:52.312448 29789 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786115480851507-30472-0/minicluster-data/master-0/data,/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786115480851507-30472-0/minicluster-data/master-0/wal
uuid: "6652f599c801480e98e8e7b3c5a95c21"
format_stamp: "Formatted at 2026-08-07 15:15:52 on dist-test-slave-42z9"
server_key: "3c597f827f82d4cb9e4d9bb35c8cadd9"
server_key_iv: "49af32a1027c5ead6035f074683f5618"
server_key_version: "encryptionkey@0"
I20260807 15:15:52.312566 29789 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786115480851507-30472-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786115480851507-30472-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786115480851507-30472-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:15:52.352216 29789 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:15:52.354589 29789 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:15:52.354753 29789 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:15:52.358857 29789 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45555
I20260807 15:15:52.358893 29883 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45555 every 8 connection(s)
I20260807 15:15:52.359297 29789 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786115480851507-30472-0/minicluster-data/master-0/data/info.pb
I20260807 15:15:52.360381 29884 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:15:52.362939 29884 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6652f599c801480e98e8e7b3c5a95c21: Bootstrap starting.
I20260807 15:15:52.363433 30472 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7fZSIL/build/release/bin/kudu as pid 29789
I20260807 15:15:52.363562 30472 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786115480851507-30472-0/minicluster-data/master-0/wal/instance
I20260807 15:15:52.363682 29884 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6652f599c801480e98e8e7b3c5a95c21: Neither blocks nor log segments found. Creating new log.
I20260807 15:15:52.363775 30472 external_mini_cluster.cc:1428] Setting key 167355a855a8fee1b467b19976a687f3
I20260807 15:15:52.364004 29884 log.cc:824] T 00000000000000000000000000000000 P 6652f599c801480e98e8e7b3c5a95c21: Log is configured to *not* fsync() on all Append() calls
I20260807 15:15:52.364889 29884 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6652f599c801480e98e8e7b3c5a95c21: No bootstrap required, opened a new log
I20260807 15:15:52.367146 29884 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6652f599c801480e98e8e7b3c5a95c21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6652f599c801480e98e8e7b3c5a95c21" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45555 } }
I20260807 15:15:52.367341 29884 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6652f599c801480e98e8e7b3c5a95c21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:15:52.367372 29884 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6652f599c801480e98e8e7b3c5a95c21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6652f599c801480e98e8e7b3c5a95c21, State: Initialized, Role: FOLLOWER
I20260807 15:15:52.367527 29884 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6652f599c801480e98e8e7b3c5a95c21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6652f599c801480e98e8e7b3c5a95c21" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45555 } }
I20260807 15:15:52.367616 29884 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6652f599c801480e98e8e7b3c5a95c21 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:15:52.367658 29884 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6652f599c801480e98e8e7b3c5a95c21 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:15:52.367704 29884 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6652f599c801480e98e8e7b3c5a95c21 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:15:52.368507 29884 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6652f599c801480e98e8e7b3c5a95c21 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6652f599c801480e98e8e7b3c5a95c21" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45555 } }
I20260807 15:15:52.368606 29884 leader_election.cc:302] T 00000000000000000000000000000000 P 6652f599c801480e98e8e7b3c5a95c21 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6652f599c801480e98e8e7b3c5a95c21; no voters:
I20260807 15:15:52.368759 29884 leader_election.cc:288] T 00000000000000000000000000000000 P 6652f599c801480e98e8e7b3c5a95c21 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 15:15:52.368842 29895 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6652f599c801480e98e8e7b3c5a95c21 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:15:52.369059 29884 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6652f599c801480e98e8e7b3c5a95c21 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:15:52.369385 29895 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6652f599c801480e98e8e7b3c5a95c21 [term 1 LEADER]: Becoming Leader. State: Replica: 6652f599c801480e98e8e7b3c5a95c21, State: Running, Role: LEADER
I20260807 15:15:52.369565 29895 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6652f599c801480e98e8e7b3c5a95c21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6652f599c801480e98e8e7b3c5a95c21" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45555 } }
I20260807 15:15:52.369956 29896 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6652f599c801480e98e8e7b3c5a95c21 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6652f599c801480e98e8e7b3c5a95c21" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6652f599c801480e98e8e7b3c5a95c21" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45555 } } }
I20260807 15:15:52.370064 29896 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6652f599c801480e98e8e7b3c5a95c21 [sys.catalog]: This master's current role is: LEADER
I20260807 15:15:52.370275 29894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:52.364353 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.33:52243 (local address 127.0.0.1:45555)
0807 15:15:52.364540 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:52.364550 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:52.364589 (+ 39us) server_negotiation.cc:407] Connection header received
0807 15:15:52.365110 (+ 521us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:52.365119 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:52.365214 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:52.365338 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:15:52.365605 (+ 267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:52.366472 (+ 867us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:52.367599 (+ 1127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:52.368203 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:52.368332 (+ 129us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:52.369464 (+ 1132us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:15:52.369468 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:15:52.369469 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:15:52.369687 (+ 218us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:15:52.369735 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:15:52.369752 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:15:52.369840 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0807 15:15:52.370043 (+ 203us) server_negotiation.cc:299] Negotiation successful
0807 15:15:52.370074 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":128,"thread_start_us":72,"threads_started":1}
I20260807 15:15:52.369968 29898 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6652f599c801480e98e8e7b3c5a95c21 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6652f599c801480e98e8e7b3c5a95c21. Latest consensus state: current_term: 1 leader_uuid: "6652f599c801480e98e8e7b3c5a95c21" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6652f599c801480e98e8e7b3c5a95c21" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45555 } } }
I20260807 15:15:52.370365 29898 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6652f599c801480e98e8e7b3c5a95c21 [sys.catalog]: This master's current role is: LEADER
I20260807 15:15:52.370625 29907 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:15:52.371574 29907 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:15:52.371574 30472 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 15:15:52.373124 29907 catalog_manager.cc:1347] Generated new cluster ID: db92b58852d44d21a9457b17b92947ab
I20260807 15:15:52.373183 29907 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:15:52.377118 29894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:52.373091 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.33:45219 (local address 127.0.0.1:45555)
0807 15:15:52.373163 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:52.373165 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:52.373653 (+ 488us) server_negotiation.cc:407] Connection header received
0807 15:15:52.373714 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:52.373715 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:52.373726 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:52.373783 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:15:52.373914 (+ 131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:52.374680 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:52.375745 (+ 1065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:52.376056 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:52.376111 (+ 55us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:52.376856 (+ 745us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:15:52.376857 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:15:52.376858 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:15:52.376862 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:15:52.376879 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:15:52.376883 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:15:52.376934 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0807 15:15:52.377016 (+ 82us) server_negotiation.cc:299] Negotiation successful
0807 15:15:52.377034 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260807 15:15:52.393990 29907 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:15:52.394759 29907 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:15:52.411525 29907 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6652f599c801480e98e8e7b3c5a95c21: Generated new TSK 0
I20260807 15:15:52.411754 29907 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 15:15:52.421751 30472 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7fZSIL/build/release/bin/kudu with pid 29789
W20260807 15:15:52.434252 29766 connection.cc:535] client connection to 127.0.0.1:45555 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:45555): Connection reset by peer (error 104)
2026-08-07T15:15:52Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1522 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786115480851507-30472-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 15:15:52 dist-test-slave-42z9 krb5kdc[29928](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 15:15:52 dist-test-slave-42z9 krb5kdc[29928](info): set up 2 sockets
Aug 07 15:15:52 dist-test-slave-42z9 krb5kdc[29928](info): commencing operation
krb5kdc: starting...
W20260807 15:15:53.576891 30472 mini_kdc.cc:121] Time spent starting KDC: real 1.129s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786115480851507-30472-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786115480851507-30472-0/krb5kdc/test-user.keytab.
Aug 07 15:15:53 dist-test-slave-42z9 krb5kdc[29928](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115753, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T15:15:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T15:15:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786115480851507-30472-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786115480851507-30472-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 15:15:53.708623 30472 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7fZSIL/build/release/bin/kudu
/tmp/dist-test-task7fZSIL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786115480851507-30472-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786115480851507-30472-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786115480851507-30472-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786115480851507-30472-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41351
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786115480851507-30472-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38011
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41351
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786115480851507-30472-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786115480851507-30472-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786115480851507-30472-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:15:53.795545 29945 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:15:53.795789 29945 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:15:53.795814 29945 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:15:53.797035 29945 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:15:53.797079 29945 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:15:53.797092 29945 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:15:53.797103 29945 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:15:53.797113 29945 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:15:53.798839 29945 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38011
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786115480851507-30472-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786115480851507-30472-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41351
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786115480851507-30472-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41351
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786115480851507-30472-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29945
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786115480851507-30472-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:15:53.799106 29945 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:15:53.799309 29945 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:15:53.804833 29954 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:15:53.805047 29945 server_base.cc:1034] running on GCE node
W20260807 15:15:53.804859 29957 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:15:53.805208 29955 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:15:53.805662 29945 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:15:53.806049 29945 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:15:53.807240 29945 hybrid_clock.cc:648] HybridClock initialized: now 1786115753807186 us; error 67 us; skew 500 ppm
Aug 07 15:15:53 dist-test-slave-42z9 krb5kdc[29928](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115753, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:15:53.810154 29945 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 15:15:53.810659 29945 webserver.cc:460] Webserver started at http://127.0.0.1:45669/ using document root <none> and password file <none>
I20260807 15:15:53.810907 29945 fs_manager.cc:359] Metadata directory not provided
I20260807 15:15:53.810956 29945 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:15:53.811080 29945 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:15:53.812192 29945 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786115480851507-30472-0/minicluster-data/master-0/data/instance:
uuid: "13369d7004cc4333b2fa87284250b1a4"
format_stamp: "Formatted at 2026-08-07 15:15:53 on dist-test-slave-42z9"
server_key: "6c0fe6b4ec9869361cb1a10248b51bbb"
server_key_iv: "ccc73906214e114048b1cb4e7d3540bf"
server_key_version: "encryptionkey@0"
I20260807 15:15:53.812525 29945 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786115480851507-30472-0/minicluster-data/master-0/wal/instance:
uuid: "13369d7004cc4333b2fa87284250b1a4"
format_stamp: "Formatted at 2026-08-07 15:15:53 on dist-test-slave-42z9"
server_key: "6c0fe6b4ec9869361cb1a10248b51bbb"
server_key_iv: "ccc73906214e114048b1cb4e7d3540bf"
server_key_version: "encryptionkey@0"
I20260807 15:15:53.813925 29945 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 15:15:53.814800 29970 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:15:53.815096 29945 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:15:53.815186 29945 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786115480851507-30472-0/minicluster-data/master-0/data,/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786115480851507-30472-0/minicluster-data/master-0/wal
uuid: "13369d7004cc4333b2fa87284250b1a4"
format_stamp: "Formatted at 2026-08-07 15:15:53 on dist-test-slave-42z9"
server_key: "6c0fe6b4ec9869361cb1a10248b51bbb"
server_key_iv: "ccc73906214e114048b1cb4e7d3540bf"
server_key_version: "encryptionkey@0"
I20260807 15:15:53.815263 29945 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786115480851507-30472-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786115480851507-30472-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786115480851507-30472-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:15:53.818822 29945 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:15:53.821022 29945 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:15:53.821153 29945 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:15:53.826251 29945 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41351
I20260807 15:15:53.826335 30057 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41351 every 8 connection(s)
I20260807 15:15:53.826634 29945 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786115480851507-30472-0/minicluster-data/master-0/data/info.pb
I20260807 15:15:53.827705 30058 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:15:53.830256 30058 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 13369d7004cc4333b2fa87284250b1a4: Bootstrap starting.
I20260807 15:15:53.830894 30058 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 13369d7004cc4333b2fa87284250b1a4: Neither blocks nor log segments found. Creating new log.
I20260807 15:15:53.831199 30058 log.cc:824] T 00000000000000000000000000000000 P 13369d7004cc4333b2fa87284250b1a4: Log is configured to *not* fsync() on all Append() calls
I20260807 15:15:53.832075 30058 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 13369d7004cc4333b2fa87284250b1a4: No bootstrap required, opened a new log
I20260807 15:15:53.833832 30058 raft_consensus.cc:348] T 00000000000000000000000000000000 P 13369d7004cc4333b2fa87284250b1a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "13369d7004cc4333b2fa87284250b1a4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41351 } }
I20260807 15:15:53.833987 30058 raft_consensus.cc:374] T 00000000000000000000000000000000 P 13369d7004cc4333b2fa87284250b1a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:15:53.834009 30058 raft_consensus.cc:729] T 00000000000000000000000000000000 P 13369d7004cc4333b2fa87284250b1a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 13369d7004cc4333b2fa87284250b1a4, State: Initialized, Role: FOLLOWER
I20260807 15:15:53.834165 30058 consensus_queue.cc:260] T 00000000000000000000000000000000 P 13369d7004cc4333b2fa87284250b1a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "13369d7004cc4333b2fa87284250b1a4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41351 } }
I20260807 15:15:53.834232 30058 raft_consensus.cc:388] T 00000000000000000000000000000000 P 13369d7004cc4333b2fa87284250b1a4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:15:53.834255 30058 raft_consensus.cc:482] T 00000000000000000000000000000000 P 13369d7004cc4333b2fa87284250b1a4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:15:53.834302 30058 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 13369d7004cc4333b2fa87284250b1a4 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:15:53.834771 30472 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7fZSIL/build/release/bin/kudu as pid 29945
I20260807 15:15:53.834887 30472 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786115480851507-30472-0/minicluster-data/master-0/wal/instance
I20260807 15:15:53.834933 30058 raft_consensus.cc:504] T 00000000000000000000000000000000 P 13369d7004cc4333b2fa87284250b1a4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "13369d7004cc4333b2fa87284250b1a4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41351 } }
I20260807 15:15:53.835078 30472 external_mini_cluster.cc:1428] Setting key 4625cc9ec6b2431c369b8b28629f3191
I20260807 15:15:53.835050 30058 leader_election.cc:302] T 00000000000000000000000000000000 P 13369d7004cc4333b2fa87284250b1a4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 13369d7004cc4333b2fa87284250b1a4; no voters:
I20260807 15:15:53.835243 30058 leader_election.cc:288] T 00000000000000000000000000000000 P 13369d7004cc4333b2fa87284250b1a4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 15:15:53.835290 30063 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 13369d7004cc4333b2fa87284250b1a4 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:15:53.835412 30063 raft_consensus.cc:686] T 00000000000000000000000000000000 P 13369d7004cc4333b2fa87284250b1a4 [term 1 LEADER]: Becoming Leader. State: Replica: 13369d7004cc4333b2fa87284250b1a4, State: Running, Role: LEADER
I20260807 15:15:53.835536 30058 sys_catalog.cc:564] T 00000000000000000000000000000000 P 13369d7004cc4333b2fa87284250b1a4 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:15:53.835567 30063 consensus_queue.cc:237] T 00000000000000000000000000000000 P 13369d7004cc4333b2fa87284250b1a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "13369d7004cc4333b2fa87284250b1a4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41351 } }
I20260807 15:15:53.836015 30065 sys_catalog.cc:455] T 00000000000000000000000000000000 P 13369d7004cc4333b2fa87284250b1a4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 13369d7004cc4333b2fa87284250b1a4. Latest consensus state: current_term: 1 leader_uuid: "13369d7004cc4333b2fa87284250b1a4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "13369d7004cc4333b2fa87284250b1a4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41351 } } }
I20260807 15:15:53.836102 30065 sys_catalog.cc:458] T 00000000000000000000000000000000 P 13369d7004cc4333b2fa87284250b1a4 [sys.catalog]: This master's current role is: LEADER
I20260807 15:15:53.836236 30066 sys_catalog.cc:455] T 00000000000000000000000000000000 P 13369d7004cc4333b2fa87284250b1a4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "13369d7004cc4333b2fa87284250b1a4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "13369d7004cc4333b2fa87284250b1a4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41351 } } }
I20260807 15:15:53.836279 30066 sys_catalog.cc:458] T 00000000000000000000000000000000 P 13369d7004cc4333b2fa87284250b1a4 [sys.catalog]: This master's current role is: LEADER
I20260807 15:15:53.836720 30075 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:15:53.837363 30075 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:15:53.837872 30073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:53.835768 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.33:59505 (local address 127.0.0.1:41351)
0807 15:15:53.836010 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:53.836019 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:53.836052 (+ 33us) server_negotiation.cc:407] Connection header received
0807 15:15:53.836176 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:53.836184 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:53.836216 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:53.836665 (+ 449us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:15:53.837228 (+ 563us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:15:53.837231 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:15:53.837231 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:15:53.837235 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:15:53.837276 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:15:53.837290 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:15:53.837352 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0807 15:15:53.837574 (+ 222us) server_negotiation.cc:299] Negotiation successful
0807 15:15:53.837621 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":175,"thread_start_us":69,"threads_started":1}
I20260807 15:15:53.838984 30472 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 15:15:53.839099 30075 catalog_manager.cc:1347] Generated new cluster ID: de10009ab0f040c9879ca4e43923ffe7
I20260807 15:15:53.839133 30075 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:15:53.841768 30073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:53.840513 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.33:33855 (local address 127.0.0.1:41351)
0807 15:15:53.840591 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:53.840593 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:53.840711 (+ 118us) server_negotiation.cc:407] Connection header received
0807 15:15:53.840767 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:53.840768 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:53.840774 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:53.840825 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:15:53.841528 (+ 703us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:15:53.841530 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:15:53.841530 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:15:53.841534 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:15:53.841551 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:15:53.841556 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:15:53.841611 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0807 15:15:53.841677 (+ 66us) server_negotiation.cc:299] Negotiation successful
0807 15:15:53.841690 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260807 15:15:53.846251 30075 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:15:53.846423 30075 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:15:53.849812 30075 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 13369d7004cc4333b2fa87284250b1a4: Generated new TSK 0
I20260807 15:15:53.849980 30075 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 15:15:53.854810 30472 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7fZSIL/build/release/bin/kudu with pid 29945
2026-08-07T15:15:53Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1438 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786115480851507-30472-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 15:15:53 dist-test-slave-42z9 krb5kdc[30103](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 15:15:53 dist-test-slave-42z9 krb5kdc[30103](info): set up 2 sockets
Aug 07 15:15:53 dist-test-slave-42z9 krb5kdc[30103](info): commencing operation
krb5kdc: starting...
W20260807 15:15:55.002290 30472 mini_kdc.cc:121] Time spent starting KDC: real 1.116s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786115480851507-30472-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786115480851507-30472-0/krb5kdc/test-user.keytab.
Aug 07 15:15:55 dist-test-slave-42z9 krb5kdc[30103](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115755, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T15:15:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T15:15:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.194.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.194.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.194.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.62.keytab.
Entry for principal kudu/127.29.194.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.62.keytab.
I20260807 15:15:55.137666 30472 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7fZSIL/build/release/bin/kudu
/tmp/dist-test-task7fZSIL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786115480851507-30472-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786115480851507-30472-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786115480851507-30472-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786115480851507-30472-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.194.62:37577
--webserver_interface=127.29.194.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.62.keytab
--principal=kudu/127.29.194.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.194.20:44811
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.194.62:37577
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786115480851507-30472-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786115480851507-30472-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786115480851507-30472-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:15:55.227414 30120 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:15:55.227653 30120 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:15:55.227685 30120 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:15:55.228904 30120 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:15:55.228953 30120 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:15:55.228972 30120 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:15:55.228988 30120 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:15:55.229003 30120 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:15:55.230654 30120 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.194.20:44811
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786115480851507-30472-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786115480851507-30472-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.194.62:37577
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.62.keytab
--principal=kudu/127.29.194.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.194.62:37577
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786115480851507-30472-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.194.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30120
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786115480851507-30472-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:15:55.230938 30120 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:15:55.231182 30120 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:15:55.236565 30135 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:15:55.236610 30133 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:15:55.236565 30132 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:15:55.236858 30120 server_base.cc:1034] running on GCE node
I20260807 15:15:55.237118 30120 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:15:55.237433 30120 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:15:55.238648 30120 hybrid_clock.cc:648] HybridClock initialized: now 1786115755238627 us; error 40 us; skew 500 ppm
Aug 07 15:15:55 dist-test-slave-42z9 krb5kdc[30103](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115755, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.194.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:15:55.246876 30120 init.cc:377] Logged in from keytab as kudu/127.29.194.62@KRBTEST.COM (short username kudu)
I20260807 15:15:55.247363 30120 webserver.cc:460] Webserver started at http://127.29.194.62:36511/ using document root <none> and password file <none>
I20260807 15:15:55.247613 30120 fs_manager.cc:359] Metadata directory not provided
I20260807 15:15:55.247666 30120 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:15:55.247799 30120 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:15:55.248900 30120 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786115480851507-30472-0/minicluster-data/master-0/data/instance:
uuid: "f8a740cf6ba441e59f767ded32d65de3"
format_stamp: "Formatted at 2026-08-07 15:15:55 on dist-test-slave-42z9"
server_key: "90c9a56aaef9e99f81d34ffbd7f40719"
server_key_iv: "22c2c970b278ec944d399b76867792c8"
server_key_version: "encryptionkey@0"
I20260807 15:15:55.249233 30120 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786115480851507-30472-0/minicluster-data/master-0/wal/instance:
uuid: "f8a740cf6ba441e59f767ded32d65de3"
format_stamp: "Formatted at 2026-08-07 15:15:55 on dist-test-slave-42z9"
server_key: "90c9a56aaef9e99f81d34ffbd7f40719"
server_key_iv: "22c2c970b278ec944d399b76867792c8"
server_key_version: "encryptionkey@0"
I20260807 15:15:55.250684 30120 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 15:15:55.251487 30147 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:15:55.251713 30120 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:15:55.251811 30120 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786115480851507-30472-0/minicluster-data/master-0/data,/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786115480851507-30472-0/minicluster-data/master-0/wal
uuid: "f8a740cf6ba441e59f767ded32d65de3"
format_stamp: "Formatted at 2026-08-07 15:15:55 on dist-test-slave-42z9"
server_key: "90c9a56aaef9e99f81d34ffbd7f40719"
server_key_iv: "22c2c970b278ec944d399b76867792c8"
server_key_version: "encryptionkey@0"
I20260807 15:15:55.251885 30120 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786115480851507-30472-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786115480851507-30472-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786115480851507-30472-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:15:55.269867 30120 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:15:55.272462 30120 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:15:55.272632 30120 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:15:55.276960 30120 rpc_server.cc:307] RPC server started. Bound to: 127.29.194.62:37577
I20260807 15:15:55.277052 30227 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.194.62:37577 every 8 connection(s)
I20260807 15:15:55.277386 30120 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786115480851507-30472-0/minicluster-data/master-0/data/info.pb
I20260807 15:15:55.278321 30228 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:15:55.280750 30228 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f8a740cf6ba441e59f767ded32d65de3: Bootstrap starting.
I20260807 15:15:55.281378 30228 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f8a740cf6ba441e59f767ded32d65de3: Neither blocks nor log segments found. Creating new log.
I20260807 15:15:55.281694 30228 log.cc:824] T 00000000000000000000000000000000 P f8a740cf6ba441e59f767ded32d65de3: Log is configured to *not* fsync() on all Append() calls
I20260807 15:15:55.282598 30228 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f8a740cf6ba441e59f767ded32d65de3: No bootstrap required, opened a new log
I20260807 15:15:55.284432 30472 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7fZSIL/build/release/bin/kudu as pid 30120
I20260807 15:15:55.284596 30472 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786115480851507-30472-0/minicluster-data/master-0/wal/instance
I20260807 15:15:55.284524 30228 raft_consensus.cc:348] T 00000000000000000000000000000000 P f8a740cf6ba441e59f767ded32d65de3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f8a740cf6ba441e59f767ded32d65de3" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 37577 } }
I20260807 15:15:55.284694 30228 raft_consensus.cc:374] T 00000000000000000000000000000000 P f8a740cf6ba441e59f767ded32d65de3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:15:55.284754 30228 raft_consensus.cc:729] T 00000000000000000000000000000000 P f8a740cf6ba441e59f767ded32d65de3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f8a740cf6ba441e59f767ded32d65de3, State: Initialized, Role: FOLLOWER
I20260807 15:15:55.284893 30472 external_mini_cluster.cc:1428] Setting key bae38f4084d3c3b5abf965d1fdde2d33
I20260807 15:15:55.284914 30228 consensus_queue.cc:260] T 00000000000000000000000000000000 P f8a740cf6ba441e59f767ded32d65de3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f8a740cf6ba441e59f767ded32d65de3" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 37577 } }
I20260807 15:15:55.285008 30228 raft_consensus.cc:388] T 00000000000000000000000000000000 P f8a740cf6ba441e59f767ded32d65de3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:15:55.285051 30228 raft_consensus.cc:482] T 00000000000000000000000000000000 P f8a740cf6ba441e59f767ded32d65de3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:15:55.285153 30228 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f8a740cf6ba441e59f767ded32d65de3 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:15:55.285975 30228 raft_consensus.cc:504] T 00000000000000000000000000000000 P f8a740cf6ba441e59f767ded32d65de3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f8a740cf6ba441e59f767ded32d65de3" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 37577 } }
I20260807 15:15:55.286092 30228 leader_election.cc:302] T 00000000000000000000000000000000 P f8a740cf6ba441e59f767ded32d65de3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f8a740cf6ba441e59f767ded32d65de3; no voters:
I20260807 15:15:55.286296 30228 leader_election.cc:288] T 00000000000000000000000000000000 P f8a740cf6ba441e59f767ded32d65de3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 15:15:55.286343 30239 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f8a740cf6ba441e59f767ded32d65de3 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:15:55.286458 30239 raft_consensus.cc:686] T 00000000000000000000000000000000 P f8a740cf6ba441e59f767ded32d65de3 [term 1 LEADER]: Becoming Leader. State: Replica: f8a740cf6ba441e59f767ded32d65de3, State: Running, Role: LEADER
I20260807 15:15:55.286638 30228 sys_catalog.cc:564] T 00000000000000000000000000000000 P f8a740cf6ba441e59f767ded32d65de3 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:15:55.286623 30239 consensus_queue.cc:237] T 00000000000000000000000000000000 P f8a740cf6ba441e59f767ded32d65de3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f8a740cf6ba441e59f767ded32d65de3" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 37577 } }
I20260807 15:15:55.286978 30239 sys_catalog.cc:455] T 00000000000000000000000000000000 P f8a740cf6ba441e59f767ded32d65de3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f8a740cf6ba441e59f767ded32d65de3. Latest consensus state: current_term: 1 leader_uuid: "f8a740cf6ba441e59f767ded32d65de3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f8a740cf6ba441e59f767ded32d65de3" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 37577 } } }
I20260807 15:15:55.287063 30239 sys_catalog.cc:458] T 00000000000000000000000000000000 P f8a740cf6ba441e59f767ded32d65de3 [sys.catalog]: This master's current role is: LEADER
I20260807 15:15:55.287328 30240 sys_catalog.cc:455] T 00000000000000000000000000000000 P f8a740cf6ba441e59f767ded32d65de3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f8a740cf6ba441e59f767ded32d65de3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f8a740cf6ba441e59f767ded32d65de3" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 37577 } } }
I20260807 15:15:55.287458 30240 sys_catalog.cc:458] T 00000000000000000000000000000000 P f8a740cf6ba441e59f767ded32d65de3 [sys.catalog]: This master's current role is: LEADER
W20260807 15:15:55.288625 30260 catalog_manager.cc:1558] T 00000000000000000000000000000000 P f8a740cf6ba441e59f767ded32d65de3: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 15:15:55.288689 30260 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 15:15:55.288731 30246 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:15:55.288882 30246 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:15:55.290704 30246 catalog_manager.cc:1347] Generated new cluster ID: 60794581d53d4ba8bd92f97567378d7b
I20260807 15:15:55.290771 30246 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 07 15:15:55 dist-test-slave-42z9 krb5kdc[30103](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115755, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.194.62@KRBTEST.COM
I20260807 15:15:55.297793 30238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:55.285588 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44735 (local address 127.29.194.62:37577)
0807 15:15:55.286202 (+ 614us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:55.286216 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:55.286264 (+ 48us) server_negotiation.cc:407] Connection header received
0807 15:15:55.286440 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:55.286449 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:55.286531 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:55.287724 (+ 1193us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:15:55.287886 (+ 162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:55.289236 (+ 1350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:55.290466 (+ 1230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:55.291085 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:55.291207 (+ 122us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:55.293843 (+ 2636us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:15:55.293847 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:15:55.293848 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:15:55.293860 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:15:55.296066 (+ 2206us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:15:55.296500 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:15:55.296504 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:15:55.296504 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:15:55.296618 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:15:55.296855 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:15:55.296855 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:15:55.296855 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:15:55.297120 (+ 265us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:15:55.297298 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0807 15:15:55.297425 (+ 127us) server_negotiation.cc:299] Negotiation successful
0807 15:15:55.297598 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":539,"thread_start_us":451,"threads_started":1}
I20260807 15:15:55.298959 30472 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 15:15:55.299098 30246 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:15:55.299778 30246 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:15:55.307703 30246 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f8a740cf6ba441e59f767ded32d65de3: Generated new TSK 0
I20260807 15:15:55.307914 30246 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 15:15:55.308336 30238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:55.300625 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36513 (local address 127.29.194.62:37577)
0807 15:15:55.300666 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:55.300668 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:55.300678 (+ 10us) server_negotiation.cc:407] Connection header received
0807 15:15:55.300736 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:55.300737 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:55.300747 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:55.300803 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:15:55.301431 (+ 628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:55.302270 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:55.303312 (+ 1042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:55.303662 (+ 350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:55.303737 (+ 75us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:55.305165 (+ 1428us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:15:55.305166 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:15:55.305167 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:15:55.305175 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:15:55.306766 (+ 1591us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:15:55.307207 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:15:55.307208 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:15:55.307209 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:15:55.307266 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:15:55.307515 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:15:55.307516 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:15:55.307517 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:15:55.307648 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:15:55.307726 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0807 15:15:55.308051 (+ 325us) server_negotiation.cc:299] Negotiation successful
0807 15:15:55.308244 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260807 15:15:55.310361 30472 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7fZSIL/build/release/bin/kudu with pid 30120
W20260807 15:15:55.325326 30101 connection.cc:535] client connection to 127.29.194.62:37577 recv error: Network error: failed to read from TLS socket (remote: 127.29.194.62:37577): Connection reset by peer (error 104)
2026-08-07T15:15:55Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1452 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786115480851507-30472-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 15:15:55 dist-test-slave-42z9 krb5kdc[30275](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 15:15:55 dist-test-slave-42z9 krb5kdc[30275](info): set up 2 sockets
Aug 07 15:15:55 dist-test-slave-42z9 krb5kdc[30275](info): commencing operation
krb5kdc: starting...
W20260807 15:15:56.477428 30472 mini_kdc.cc:121] Time spent starting KDC: real 1.138s user 0.006s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786115480851507-30472-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786115480851507-30472-0/krb5kdc/test-user.keytab.
Aug 07 15:15:56 dist-test-slave-42z9 krb5kdc[30275](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115756, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T15:15:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T15:15:56Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.194.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.194.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.194.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.62.keytab.
Entry for principal kudu/127.29.194.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.62.keytab.
I20260807 15:15:56.613897 30472 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7fZSIL/build/release/bin/kudu
/tmp/dist-test-task7fZSIL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786115480851507-30472-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786115480851507-30472-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786115480851507-30472-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786115480851507-30472-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.194.62:43027
--webserver_interface=127.29.194.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.62.keytab
--principal=kudu/127.29.194.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.194.20:34719
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.194.62:43027
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786115480851507-30472-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786115480851507-30472-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786115480851507-30472-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:15:56.698601 30293 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:15:56.698836 30293 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:15:56.698861 30293 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:15:56.700174 30293 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:15:56.700233 30293 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:15:56.700248 30293 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:15:56.700258 30293 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:15:56.700269 30293 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:15:56.701923 30293 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.194.20:34719
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786115480851507-30472-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786115480851507-30472-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.194.62:43027
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.62.keytab
--principal=kudu/127.29.194.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.194.62:43027
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786115480851507-30472-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.194.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30293
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786115480851507-30472-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:15:56.702186 30293 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:15:56.702423 30293 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:15:56.707860 30304 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:15:56.707840 30300 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:15:56.707963 30293 server_base.cc:1034] running on GCE node
W20260807 15:15:56.707840 30302 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:15:56.708699 30293 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:15:56.709116 30293 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:15:56.710310 30293 hybrid_clock.cc:648] HybridClock initialized: now 1786115756710281 us; error 55 us; skew 500 ppm
Aug 07 15:15:56 dist-test-slave-42z9 krb5kdc[30275](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115756, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.194.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:15:56.713202 30293 init.cc:377] Logged in from keytab as kudu/127.29.194.62@KRBTEST.COM (short username kudu)
I20260807 15:15:56.713775 30293 webserver.cc:460] Webserver started at http://127.29.194.62:37573/ using document root <none> and password file <none>
I20260807 15:15:56.714001 30293 fs_manager.cc:359] Metadata directory not provided
I20260807 15:15:56.714053 30293 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:15:56.714183 30293 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:15:56.715281 30293 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786115480851507-30472-0/minicluster-data/master-0/data/instance:
uuid: "d415dfb1c7544bbf837738a9d715c3b3"
format_stamp: "Formatted at 2026-08-07 15:15:56 on dist-test-slave-42z9"
server_key: "9f3bc866380126b50c7a0436a9d5f6f7"
server_key_iv: "ebfad966a422f79ae08f3fb6b28ef5af"
server_key_version: "encryptionkey@0"
I20260807 15:15:56.715631 30293 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786115480851507-30472-0/minicluster-data/master-0/wal/instance:
uuid: "d415dfb1c7544bbf837738a9d715c3b3"
format_stamp: "Formatted at 2026-08-07 15:15:56 on dist-test-slave-42z9"
server_key: "9f3bc866380126b50c7a0436a9d5f6f7"
server_key_iv: "ebfad966a422f79ae08f3fb6b28ef5af"
server_key_version: "encryptionkey@0"
I20260807 15:15:56.716995 30293 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 15:15:56.718058 30313 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:15:56.718431 30293 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:15:56.718547 30293 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786115480851507-30472-0/minicluster-data/master-0/data,/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786115480851507-30472-0/minicluster-data/master-0/wal
uuid: "d415dfb1c7544bbf837738a9d715c3b3"
format_stamp: "Formatted at 2026-08-07 15:15:56 on dist-test-slave-42z9"
server_key: "9f3bc866380126b50c7a0436a9d5f6f7"
server_key_iv: "ebfad966a422f79ae08f3fb6b28ef5af"
server_key_version: "encryptionkey@0"
I20260807 15:15:56.718626 30293 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786115480851507-30472-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786115480851507-30472-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786115480851507-30472-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:15:56.736624 30293 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:15:56.739356 30293 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:15:56.739529 30293 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:15:56.743808 30293 rpc_server.cc:307] RPC server started. Bound to: 127.29.194.62:43027
I20260807 15:15:56.743906 30401 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.194.62:43027 every 8 connection(s)
I20260807 15:15:56.744228 30293 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786115480851507-30472-0/minicluster-data/master-0/data/info.pb
I20260807 15:15:56.745178 30404 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:15:56.747680 30404 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d415dfb1c7544bbf837738a9d715c3b3: Bootstrap starting.
I20260807 15:15:56.748333 30404 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d415dfb1c7544bbf837738a9d715c3b3: Neither blocks nor log segments found. Creating new log.
I20260807 15:15:56.748620 30404 log.cc:824] T 00000000000000000000000000000000 P d415dfb1c7544bbf837738a9d715c3b3: Log is configured to *not* fsync() on all Append() calls
I20260807 15:15:56.749400 30404 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d415dfb1c7544bbf837738a9d715c3b3: No bootstrap required, opened a new log
I20260807 15:15:56.750453 30472 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7fZSIL/build/release/bin/kudu as pid 30293
I20260807 15:15:56.750617 30472 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786115480851507-30472-0/minicluster-data/master-0/wal/instance
I20260807 15:15:56.750857 30472 external_mini_cluster.cc:1428] Setting key b511e24c122b0c9f26502e1c83ffdcdd
I20260807 15:15:56.751331 30404 raft_consensus.cc:348] T 00000000000000000000000000000000 P d415dfb1c7544bbf837738a9d715c3b3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d415dfb1c7544bbf837738a9d715c3b3" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 43027 } }
I20260807 15:15:56.751559 30404 raft_consensus.cc:374] T 00000000000000000000000000000000 P d415dfb1c7544bbf837738a9d715c3b3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:15:56.751605 30404 raft_consensus.cc:729] T 00000000000000000000000000000000 P d415dfb1c7544bbf837738a9d715c3b3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d415dfb1c7544bbf837738a9d715c3b3, State: Initialized, Role: FOLLOWER
I20260807 15:15:56.751814 30404 consensus_queue.cc:260] T 00000000000000000000000000000000 P d415dfb1c7544bbf837738a9d715c3b3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d415dfb1c7544bbf837738a9d715c3b3" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 43027 } }
I20260807 15:15:56.751936 30404 raft_consensus.cc:388] T 00000000000000000000000000000000 P d415dfb1c7544bbf837738a9d715c3b3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:15:56.751973 30404 raft_consensus.cc:482] T 00000000000000000000000000000000 P d415dfb1c7544bbf837738a9d715c3b3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:15:56.752019 30404 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d415dfb1c7544bbf837738a9d715c3b3 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:15:56.753315 30404 raft_consensus.cc:504] T 00000000000000000000000000000000 P d415dfb1c7544bbf837738a9d715c3b3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d415dfb1c7544bbf837738a9d715c3b3" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 43027 } }
I20260807 15:15:56.753469 30404 leader_election.cc:302] T 00000000000000000000000000000000 P d415dfb1c7544bbf837738a9d715c3b3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d415dfb1c7544bbf837738a9d715c3b3; no voters:
I20260807 15:15:56.753664 30404 leader_election.cc:288] T 00000000000000000000000000000000 P d415dfb1c7544bbf837738a9d715c3b3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 15:15:56.753751 30413 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d415dfb1c7544bbf837738a9d715c3b3 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:15:56.753942 30404 sys_catalog.cc:564] T 00000000000000000000000000000000 P d415dfb1c7544bbf837738a9d715c3b3 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:15:56.753957 30413 raft_consensus.cc:686] T 00000000000000000000000000000000 P d415dfb1c7544bbf837738a9d715c3b3 [term 1 LEADER]: Becoming Leader. State: Replica: d415dfb1c7544bbf837738a9d715c3b3, State: Running, Role: LEADER
I20260807 15:15:56.754139 30413 consensus_queue.cc:237] T 00000000000000000000000000000000 P d415dfb1c7544bbf837738a9d715c3b3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d415dfb1c7544bbf837738a9d715c3b3" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 43027 } }
I20260807 15:15:56.754513 30415 sys_catalog.cc:455] T 00000000000000000000000000000000 P d415dfb1c7544bbf837738a9d715c3b3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d415dfb1c7544bbf837738a9d715c3b3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d415dfb1c7544bbf837738a9d715c3b3" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 43027 } } }
I20260807 15:15:56.754758 30415 sys_catalog.cc:458] T 00000000000000000000000000000000 P d415dfb1c7544bbf837738a9d715c3b3 [sys.catalog]: This master's current role is: LEADER
I20260807 15:15:56.754695 30414 sys_catalog.cc:455] T 00000000000000000000000000000000 P d415dfb1c7544bbf837738a9d715c3b3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d415dfb1c7544bbf837738a9d715c3b3. Latest consensus state: current_term: 1 leader_uuid: "d415dfb1c7544bbf837738a9d715c3b3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d415dfb1c7544bbf837738a9d715c3b3" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 43027 } } }
I20260807 15:15:56.755012 30414 sys_catalog.cc:458] T 00000000000000000000000000000000 P d415dfb1c7544bbf837738a9d715c3b3 [sys.catalog]: This master's current role is: LEADER
I20260807 15:15:56.755264 30417 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:15:56.755869 30417 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:15:56.757907 30411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:56.751652 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33543 (local address 127.29.194.62:43027)
0807 15:15:56.751988 (+ 336us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:56.751999 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:56.752035 (+ 36us) server_negotiation.cc:407] Connection header received
0807 15:15:56.752171 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:56.752179 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:56.752240 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:56.752399 (+ 159us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:15:56.752678 (+ 279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:56.754051 (+ 1373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:56.755170 (+ 1119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:56.755804 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:56.756415 (+ 611us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:56.756765 (+ 350us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:15:56.756771 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:15:56.756771 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:15:56.757055 (+ 284us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:15:56.757103 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:15:56.757120 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:15:56.757236 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0807 15:15:56.757518 (+ 282us) server_negotiation.cc:299] Negotiation successful
0807 15:15:56.757560 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":266,"thread_start_us":182,"threads_started":1}
I20260807 15:15:56.758320 30417 catalog_manager.cc:1347] Generated new cluster ID: d712e0d3b3a64c219382cec14f731a92
I20260807 15:15:56.758368 30417 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:15:56.759039 30472 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 15:15:56.764858 30411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:56.760559 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42339 (local address 127.29.194.62:43027)
0807 15:15:56.760616 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:56.760617 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:56.760725 (+ 108us) server_negotiation.cc:407] Connection header received
0807 15:15:56.760806 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:56.760807 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:56.760816 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:56.760842 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:15:56.761140 (+ 298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:56.761950 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:56.763009 (+ 1059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:56.763346 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:56.763384 (+ 38us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:56.764590 (+ 1206us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:15:56.764592 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:15:56.764592 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:15:56.764597 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:15:56.764614 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:15:56.764620 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:15:56.764649 (+ 29us) server_negotiation.cc:1035] Waiting for connection context
0807 15:15:56.764758 (+ 109us) server_negotiation.cc:299] Negotiation successful
0807 15:15:56.764772 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260807 15:15:56.773640 30417 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:15:56.774412 30417 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:15:56.777854 30417 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d415dfb1c7544bbf837738a9d715c3b3: Generated new TSK 0
I20260807 15:15:56.778048 30417 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 15:15:56.782320 30472 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7fZSIL/build/release/bin/kudu with pid 30293
W20260807 15:15:56.803828 30272 connection.cc:535] client connection to 127.29.194.62:43027 recv error: Network error: failed to read from TLS socket (remote: 127.29.194.62:43027): Connection reset by peer (error 104)
2026-08-07T15:15:56Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1478 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786115480851507-30472-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 15:15:56 dist-test-slave-42z9 krb5kdc[30450](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 15:15:56 dist-test-slave-42z9 krb5kdc[30450](info): set up 2 sockets
Aug 07 15:15:56 dist-test-slave-42z9 krb5kdc[30450](info): commencing operation
krb5kdc: starting...
W20260807 15:15:57.946096 30472 mini_kdc.cc:121] Time spent starting KDC: real 1.128s user 0.004s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786115480851507-30472-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786115480851507-30472-0/krb5kdc/test-user.keytab.
Aug 07 15:15:58 dist-test-slave-42z9 krb5kdc[30450](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115758, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T15:15:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T15:15:58Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.194.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.194.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.194.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.62.keytab.
Entry for principal kudu/127.29.194.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.62.keytab.
I20260807 15:15:58.077195 30472 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7fZSIL/build/release/bin/kudu
/tmp/dist-test-task7fZSIL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786115480851507-30472-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786115480851507-30472-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786115480851507-30472-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786115480851507-30472-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.194.62:33599
--webserver_interface=127.29.194.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.62.keytab
--principal=kudu/127.29.194.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.194.20:39487
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.194.62:33599
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786115480851507-30472-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786115480851507-30472-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786115480851507-30472-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:15:58.162228 30465 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:15:58.162467 30465 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:15:58.162503 30465 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:15:58.163717 30465 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:15:58.163769 30465 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:15:58.163789 30465 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:15:58.163805 30465 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:15:58.163820 30465 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:15:58.165400 30465 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.194.20:39487
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786115480851507-30472-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786115480851507-30472-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.194.62:33599
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.62.keytab
--principal=kudu/127.29.194.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.194.62:33599
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786115480851507-30472-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.194.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30465
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786115480851507-30472-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:15:58.165665 30465 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:15:58.165855 30465 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:15:58.171115 30480 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:15:58.171161 30465 server_base.cc:1034] running on GCE node
W20260807 15:15:58.171128 30486 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:15:58.171126 30481 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:15:58.171598 30465 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:15:58.171905 30465 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:15:58.173133 30465 hybrid_clock.cc:648] HybridClock initialized: now 1786115758173020 us; error 127 us; skew 500 ppm
Aug 07 15:15:58 dist-test-slave-42z9 krb5kdc[30450](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115758, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.194.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:15:58.175880 30465 init.cc:377] Logged in from keytab as kudu/127.29.194.62@KRBTEST.COM (short username kudu)
I20260807 15:15:58.176358 30465 webserver.cc:460] Webserver started at http://127.29.194.62:42243/ using document root <none> and password file <none>
I20260807 15:15:58.176568 30465 fs_manager.cc:359] Metadata directory not provided
I20260807 15:15:58.176616 30465 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:15:58.176748 30465 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:15:58.177887 30465 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786115480851507-30472-0/minicluster-data/master-0/data/instance:
uuid: "fe9e037fd43e4e848474b80fd6ff3e28"
format_stamp: "Formatted at 2026-08-07 15:15:58 on dist-test-slave-42z9"
server_key: "b09d4f46692af176ffa83630afe53412"
server_key_iv: "d2bbbdaea632f2ab09ec644091970056"
server_key_version: "encryptionkey@0"
I20260807 15:15:58.178238 30465 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786115480851507-30472-0/minicluster-data/master-0/wal/instance:
uuid: "fe9e037fd43e4e848474b80fd6ff3e28"
format_stamp: "Formatted at 2026-08-07 15:15:58 on dist-test-slave-42z9"
server_key: "b09d4f46692af176ffa83630afe53412"
server_key_iv: "d2bbbdaea632f2ab09ec644091970056"
server_key_version: "encryptionkey@0"
I20260807 15:15:58.179659 30465 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 15:15:58.180503 30494 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:15:58.180721 30465 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:15:58.180819 30465 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786115480851507-30472-0/minicluster-data/master-0/data,/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786115480851507-30472-0/minicluster-data/master-0/wal
uuid: "fe9e037fd43e4e848474b80fd6ff3e28"
format_stamp: "Formatted at 2026-08-07 15:15:58 on dist-test-slave-42z9"
server_key: "b09d4f46692af176ffa83630afe53412"
server_key_iv: "d2bbbdaea632f2ab09ec644091970056"
server_key_version: "encryptionkey@0"
I20260807 15:15:58.180886 30465 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786115480851507-30472-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786115480851507-30472-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786115480851507-30472-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:15:58.184726 30465 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:15:58.186857 30465 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:15:58.186995 30465 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:15:58.191076 30465 rpc_server.cc:307] RPC server started. Bound to: 127.29.194.62:33599
I20260807 15:15:58.191130 30572 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.194.62:33599 every 8 connection(s)
I20260807 15:15:58.191434 30465 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786115480851507-30472-0/minicluster-data/master-0/data/info.pb
I20260807 15:15:58.192487 30575 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:15:58.193248 30472 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7fZSIL/build/release/bin/kudu as pid 30465
I20260807 15:15:58.193392 30472 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786115480851507-30472-0/minicluster-data/master-0/wal/instance
I20260807 15:15:58.193647 30472 external_mini_cluster.cc:1428] Setting key 9ab7656c4300db5cd5821c1a85cf1e38
I20260807 15:15:58.195837 30575 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fe9e037fd43e4e848474b80fd6ff3e28: Bootstrap starting.
I20260807 15:15:58.196369 30579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:58.194221 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41817 (local address 127.29.194.62:33599)
0807 15:15:58.194402 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:58.194411 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:58.194441 (+ 30us) server_negotiation.cc:407] Connection header received
0807 15:15:58.194976 (+ 535us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:58.194986 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:58.195023 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:58.195191 (+ 168us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:15:58.195844 (+ 653us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:15:58.195848 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:15:58.195848 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:15:58.195853 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:15:58.195897 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:15:58.195913 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:15:58.195991 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0807 15:15:58.196110 (+ 119us) server_negotiation.cc:299] Negotiation successful
0807 15:15:58.196152 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":128,"thread_start_us":71,"threads_started":1}
I20260807 15:15:58.196734 30575 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fe9e037fd43e4e848474b80fd6ff3e28: Neither blocks nor log segments found. Creating new log.
I20260807 15:15:58.197055 30575 log.cc:824] T 00000000000000000000000000000000 P fe9e037fd43e4e848474b80fd6ff3e28: Log is configured to *not* fsync() on all Append() calls
I20260807 15:15:58.197479 30472 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 15:15:58.197815 30575 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fe9e037fd43e4e848474b80fd6ff3e28: No bootstrap required, opened a new log
I20260807 15:15:58.200251 30579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:58.198976 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50121 (local address 127.29.194.62:33599)
0807 15:15:58.199051 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:58.199053 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:58.199064 (+ 11us) server_negotiation.cc:407] Connection header received
0807 15:15:58.199119 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:58.199120 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:58.199125 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:58.199194 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:15:58.200034 (+ 840us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:15:58.200036 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:15:58.200036 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:15:58.200041 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:15:58.200061 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:15:58.200067 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:15:58.200128 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0807 15:15:58.200172 (+ 44us) server_negotiation.cc:299] Negotiation successful
0807 15:15:58.200189 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260807 15:15:58.200416 30575 raft_consensus.cc:348] T 00000000000000000000000000000000 P fe9e037fd43e4e848474b80fd6ff3e28 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe9e037fd43e4e848474b80fd6ff3e28" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 33599 } }
I20260807 15:15:58.200552 30575 raft_consensus.cc:374] T 00000000000000000000000000000000 P fe9e037fd43e4e848474b80fd6ff3e28 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:15:58.200572 30575 raft_consensus.cc:729] T 00000000000000000000000000000000 P fe9e037fd43e4e848474b80fd6ff3e28 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fe9e037fd43e4e848474b80fd6ff3e28, State: Initialized, Role: FOLLOWER
I20260807 15:15:58.200670 30575 consensus_queue.cc:260] T 00000000000000000000000000000000 P fe9e037fd43e4e848474b80fd6ff3e28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe9e037fd43e4e848474b80fd6ff3e28" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 33599 } }
I20260807 15:15:58.200750 30575 raft_consensus.cc:388] T 00000000000000000000000000000000 P fe9e037fd43e4e848474b80fd6ff3e28 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:15:58.200780 30575 raft_consensus.cc:482] T 00000000000000000000000000000000 P fe9e037fd43e4e848474b80fd6ff3e28 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:15:58.200814 30575 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fe9e037fd43e4e848474b80fd6ff3e28 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:15:58.201404 30575 raft_consensus.cc:504] T 00000000000000000000000000000000 P fe9e037fd43e4e848474b80fd6ff3e28 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe9e037fd43e4e848474b80fd6ff3e28" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 33599 } }
I20260807 15:15:58.201540 30575 leader_election.cc:302] T 00000000000000000000000000000000 P fe9e037fd43e4e848474b80fd6ff3e28 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fe9e037fd43e4e848474b80fd6ff3e28; no voters:
I20260807 15:15:58.201709 30575 leader_election.cc:288] T 00000000000000000000000000000000 P fe9e037fd43e4e848474b80fd6ff3e28 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 15:15:58.201946 30589 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fe9e037fd43e4e848474b80fd6ff3e28 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:15:58.202077 30575 sys_catalog.cc:564] T 00000000000000000000000000000000 P fe9e037fd43e4e848474b80fd6ff3e28 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:15:58.202220 30589 raft_consensus.cc:686] T 00000000000000000000000000000000 P fe9e037fd43e4e848474b80fd6ff3e28 [term 1 LEADER]: Becoming Leader. State: Replica: fe9e037fd43e4e848474b80fd6ff3e28, State: Running, Role: LEADER
I20260807 15:15:58.202414 30589 consensus_queue.cc:237] T 00000000000000000000000000000000 P fe9e037fd43e4e848474b80fd6ff3e28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe9e037fd43e4e848474b80fd6ff3e28" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 33599 } }
I20260807 15:15:58.202951 30594 sys_catalog.cc:455] T 00000000000000000000000000000000 P fe9e037fd43e4e848474b80fd6ff3e28 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fe9e037fd43e4e848474b80fd6ff3e28. Latest consensus state: current_term: 1 leader_uuid: "fe9e037fd43e4e848474b80fd6ff3e28" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe9e037fd43e4e848474b80fd6ff3e28" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 33599 } } }
I20260807 15:15:58.203125 30594 sys_catalog.cc:458] T 00000000000000000000000000000000 P fe9e037fd43e4e848474b80fd6ff3e28 [sys.catalog]: This master's current role is: LEADER
I20260807 15:15:58.203202 30590 sys_catalog.cc:455] T 00000000000000000000000000000000 P fe9e037fd43e4e848474b80fd6ff3e28 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fe9e037fd43e4e848474b80fd6ff3e28" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe9e037fd43e4e848474b80fd6ff3e28" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 33599 } } }
I20260807 15:15:58.203267 30590 sys_catalog.cc:458] T 00000000000000000000000000000000 P fe9e037fd43e4e848474b80fd6ff3e28 [sys.catalog]: This master's current role is: LEADER
W20260807 15:15:58.204151 30609 catalog_manager.cc:1558] T 00000000000000000000000000000000 P fe9e037fd43e4e848474b80fd6ff3e28: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 15:15:58.204200 30609 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 15:15:58.204303 30607 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:15:58.204466 30607 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:15:58.206408 30607 catalog_manager.cc:1347] Generated new cluster ID: e26759dff0c045ae913a9f3f9154972a
I20260807 15:15:58.206480 30607 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:15:58.218359 30607 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:15:58.218560 30607 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:15:58.225099 30607 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fe9e037fd43e4e848474b80fd6ff3e28: Generated new TSK 0
I20260807 15:15:58.225313 30607 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 15:15:58.229861 30472 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7fZSIL/build/release/bin/kudu with pid 30465
2026-08-07T15:15:58Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1440 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786115480851507-30472-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 15:15:58 dist-test-slave-42z9 krb5kdc[30619](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 15:15:58 dist-test-slave-42z9 krb5kdc[30619](info): set up 2 sockets
Aug 07 15:15:58 dist-test-slave-42z9 krb5kdc[30619](info): commencing operation
krb5kdc: starting...
W20260807 15:15:59.376396 30472 mini_kdc.cc:121] Time spent starting KDC: real 1.118s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786115480851507-30472-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786115480851507-30472-0/krb5kdc/test-user.keytab.
Aug 07 15:15:59 dist-test-slave-42z9 krb5kdc[30619](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115759, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T15:15:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T15:15:59Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.194.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.194.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.194.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.62.keytab.
Entry for principal kudu/127.29.194.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.62.keytab.
I20260807 15:15:59.506264 30472 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7fZSIL/build/release/bin/kudu
/tmp/dist-test-task7fZSIL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786115480851507-30472-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786115480851507-30472-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786115480851507-30472-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786115480851507-30472-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.194.62:34283
--webserver_interface=127.29.194.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.62.keytab
--principal=kudu/127.29.194.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.194.20:33423
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.194.62:34283
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786115480851507-30472-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786115480851507-30472-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786115480851507-30472-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:15:59.590525 30637 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:15:59.590767 30637 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:15:59.590801 30637 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:15:59.592036 30637 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:15:59.592092 30637 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:15:59.592111 30637 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:15:59.592128 30637 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:15:59.592144 30637 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:15:59.594198 30637 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.194.20:33423
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786115480851507-30472-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786115480851507-30472-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.194.62:34283
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.62.keytab
--principal=kudu/127.29.194.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.194.62:34283
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786115480851507-30472-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.194.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30637
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786115480851507-30472-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:15:59.594515 30637 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:15:59.594743 30637 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:15:59.600502 30644 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:15:59.600476 30646 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:15:59.600481 30643 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:15:59.601193 30637 server_base.cc:1034] running on GCE node
I20260807 15:15:59.601683 30637 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:15:59.602129 30637 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:15:59.603324 30637 hybrid_clock.cc:648] HybridClock initialized: now 1786115759603302 us; error 48 us; skew 500 ppm
Aug 07 15:15:59 dist-test-slave-42z9 krb5kdc[30619](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115759, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.194.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:15:59.606443 30637 init.cc:377] Logged in from keytab as kudu/127.29.194.62@KRBTEST.COM (short username kudu)
I20260807 15:15:59.606967 30637 webserver.cc:460] Webserver started at http://127.29.194.62:41363/ using document root <none> and password file <none>
I20260807 15:15:59.607208 30637 fs_manager.cc:359] Metadata directory not provided
I20260807 15:15:59.607256 30637 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:15:59.607368 30637 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:15:59.608440 30637 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786115480851507-30472-0/minicluster-data/master-0/data/instance:
uuid: "c9ed95c24ab44a9fa9ffe67c87bdd2fe"
format_stamp: "Formatted at 2026-08-07 15:15:59 on dist-test-slave-42z9"
server_key: "cb0b09257c5c730b8372f9edebe89569"
server_key_iv: "0ee0d2ceeb2cce81fbf88356fdacab36"
server_key_version: "encryptionkey@0"
I20260807 15:15:59.608770 30637 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786115480851507-30472-0/minicluster-data/master-0/wal/instance:
uuid: "c9ed95c24ab44a9fa9ffe67c87bdd2fe"
format_stamp: "Formatted at 2026-08-07 15:15:59 on dist-test-slave-42z9"
server_key: "cb0b09257c5c730b8372f9edebe89569"
server_key_iv: "0ee0d2ceeb2cce81fbf88356fdacab36"
server_key_version: "encryptionkey@0"
I20260807 15:15:59.610138 30637 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 15:15:59.610948 30655 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:15:59.611155 30637 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:15:59.611244 30637 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786115480851507-30472-0/minicluster-data/master-0/data,/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786115480851507-30472-0/minicluster-data/master-0/wal
uuid: "c9ed95c24ab44a9fa9ffe67c87bdd2fe"
format_stamp: "Formatted at 2026-08-07 15:15:59 on dist-test-slave-42z9"
server_key: "cb0b09257c5c730b8372f9edebe89569"
server_key_iv: "0ee0d2ceeb2cce81fbf88356fdacab36"
server_key_version: "encryptionkey@0"
I20260807 15:15:59.611306 30637 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786115480851507-30472-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786115480851507-30472-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786115480851507-30472-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:15:59.618211 30637 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:15:59.620187 30637 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:15:59.620329 30637 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:15:59.624392 30637 rpc_server.cc:307] RPC server started. Bound to: 127.29.194.62:34283
I20260807 15:15:59.624477 30736 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.194.62:34283 every 8 connection(s)
I20260807 15:15:59.624802 30637 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786115480851507-30472-0/minicluster-data/master-0/data/info.pb
I20260807 15:15:59.625720 30737 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:15:59.628084 30737 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c9ed95c24ab44a9fa9ffe67c87bdd2fe: Bootstrap starting.
I20260807 15:15:59.628721 30737 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c9ed95c24ab44a9fa9ffe67c87bdd2fe: Neither blocks nor log segments found. Creating new log.
I20260807 15:15:59.629007 30737 log.cc:824] T 00000000000000000000000000000000 P c9ed95c24ab44a9fa9ffe67c87bdd2fe: Log is configured to *not* fsync() on all Append() calls
I20260807 15:15:59.630250 30737 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c9ed95c24ab44a9fa9ffe67c87bdd2fe: No bootstrap required, opened a new log
I20260807 15:15:59.632310 30472 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7fZSIL/build/release/bin/kudu as pid 30637
I20260807 15:15:59.632480 30472 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786115480851507-30472-0/minicluster-data/master-0/wal/instance
I20260807 15:15:59.632735 30472 external_mini_cluster.cc:1428] Setting key e121230f56765921a958d3c7c1c2bf43
I20260807 15:15:59.632823 30737 raft_consensus.cc:348] T 00000000000000000000000000000000 P c9ed95c24ab44a9fa9ffe67c87bdd2fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9ed95c24ab44a9fa9ffe67c87bdd2fe" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 34283 } }
I20260807 15:15:59.633062 30737 raft_consensus.cc:374] T 00000000000000000000000000000000 P c9ed95c24ab44a9fa9ffe67c87bdd2fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:15:59.633126 30737 raft_consensus.cc:729] T 00000000000000000000000000000000 P c9ed95c24ab44a9fa9ffe67c87bdd2fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c9ed95c24ab44a9fa9ffe67c87bdd2fe, State: Initialized, Role: FOLLOWER
I20260807 15:15:59.633343 30737 consensus_queue.cc:260] T 00000000000000000000000000000000 P c9ed95c24ab44a9fa9ffe67c87bdd2fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9ed95c24ab44a9fa9ffe67c87bdd2fe" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 34283 } }
I20260807 15:15:59.633504 30737 raft_consensus.cc:388] T 00000000000000000000000000000000 P c9ed95c24ab44a9fa9ffe67c87bdd2fe [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:15:59.633559 30737 raft_consensus.cc:482] T 00000000000000000000000000000000 P c9ed95c24ab44a9fa9ffe67c87bdd2fe [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:15:59.633623 30737 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c9ed95c24ab44a9fa9ffe67c87bdd2fe [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:15:59.635223 30737 raft_consensus.cc:504] T 00000000000000000000000000000000 P c9ed95c24ab44a9fa9ffe67c87bdd2fe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9ed95c24ab44a9fa9ffe67c87bdd2fe" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 34283 } }
I20260807 15:15:59.635387 30737 leader_election.cc:302] T 00000000000000000000000000000000 P c9ed95c24ab44a9fa9ffe67c87bdd2fe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c9ed95c24ab44a9fa9ffe67c87bdd2fe; no voters:
I20260807 15:15:59.635563 30737 leader_election.cc:288] T 00000000000000000000000000000000 P c9ed95c24ab44a9fa9ffe67c87bdd2fe [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 15:15:59.635622 30744 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c9ed95c24ab44a9fa9ffe67c87bdd2fe [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:15:59.635843 30737 sys_catalog.cc:564] T 00000000000000000000000000000000 P c9ed95c24ab44a9fa9ffe67c87bdd2fe [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:15:59.635826 30744 raft_consensus.cc:686] T 00000000000000000000000000000000 P c9ed95c24ab44a9fa9ffe67c87bdd2fe [term 1 LEADER]: Becoming Leader. State: Replica: c9ed95c24ab44a9fa9ffe67c87bdd2fe, State: Running, Role: LEADER
I20260807 15:15:59.636073 30744 consensus_queue.cc:237] T 00000000000000000000000000000000 P c9ed95c24ab44a9fa9ffe67c87bdd2fe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9ed95c24ab44a9fa9ffe67c87bdd2fe" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 34283 } }
I20260807 15:15:59.636512 30746 sys_catalog.cc:455] T 00000000000000000000000000000000 P c9ed95c24ab44a9fa9ffe67c87bdd2fe [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c9ed95c24ab44a9fa9ffe67c87bdd2fe" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9ed95c24ab44a9fa9ffe67c87bdd2fe" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 34283 } } }
I20260807 15:15:59.636596 30746 sys_catalog.cc:458] T 00000000000000000000000000000000 P c9ed95c24ab44a9fa9ffe67c87bdd2fe [sys.catalog]: This master's current role is: LEADER
I20260807 15:15:59.636797 30747 sys_catalog.cc:455] T 00000000000000000000000000000000 P c9ed95c24ab44a9fa9ffe67c87bdd2fe [sys.catalog]: SysCatalogTable state changed. Reason: New leader c9ed95c24ab44a9fa9ffe67c87bdd2fe. Latest consensus state: current_term: 1 leader_uuid: "c9ed95c24ab44a9fa9ffe67c87bdd2fe" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9ed95c24ab44a9fa9ffe67c87bdd2fe" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 34283 } } }
I20260807 15:15:59.636919 30747 sys_catalog.cc:458] T 00000000000000000000000000000000 P c9ed95c24ab44a9fa9ffe67c87bdd2fe [sys.catalog]: This master's current role is: LEADER
I20260807 15:15:59.637280 30758 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:15:59.637910 30758 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:15:59.639549 30758 catalog_manager.cc:1347] Generated new cluster ID: 5216abba664d49f881b1b25ef696ca8e
I20260807 15:15:59.639596 30758 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 07 15:15:59 dist-test-slave-42z9 krb5kdc[30619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115759, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.194.62@KRBTEST.COM
I20260807 15:15:59.644748 30742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:59.633500 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54101 (local address 127.29.194.62:34283)
0807 15:15:59.633761 (+ 261us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:59.633774 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:59.633820 (+ 46us) server_negotiation.cc:407] Connection header received
0807 15:15:59.633983 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:59.633992 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:59.634062 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:59.634301 (+ 239us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:15:59.634987 (+ 686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:59.636347 (+ 1360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:59.637501 (+ 1154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:59.638039 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:59.638152 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:59.640956 (+ 2804us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:15:59.640961 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:15:59.640962 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:15:59.640976 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:15:59.643076 (+ 2100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:15:59.643474 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:15:59.643477 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:15:59.643477 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:15:59.643601 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:15:59.643783 (+ 182us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:15:59.643783 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:15:59.643784 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:15:59.644122 (+ 338us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:15:59.644306 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0807 15:15:59.644377 (+ 71us) server_negotiation.cc:299] Negotiation successful
0807 15:15:59.644535 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":188,"thread_start_us":129,"threads_started":1}
I20260807 15:15:59.646088 30472 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 15:15:59.654630 30742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:15:59.647652 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46417 (local address 127.29.194.62:34283)
0807 15:15:59.647689 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0807 15:15:59.647691 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:15:59.647699 (+ 8us) server_negotiation.cc:407] Connection header received
0807 15:15:59.647721 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:15:59.647721 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:15:59.647732 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:15:59.647791 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:15:59.648375 (+ 584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:59.649168 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:59.650170 (+ 1002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:15:59.650514 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:15:59.650581 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:15:59.651945 (+ 1364us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:15:59.651947 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:15:59.651947 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:15:59.651955 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:15:59.653528 (+ 1573us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:15:59.653900 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:15:59.653902 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:15:59.653902 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:15:59.653963 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:15:59.654123 (+ 160us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:15:59.654123 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:15:59.654124 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:15:59.654256 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:15:59.654360 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0807 15:15:59.654448 (+ 88us) server_negotiation.cc:299] Negotiation successful
0807 15:15:59.654529 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260807 15:15:59.658730 30758 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:15:59.659292 30758 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:15:59.666044 30758 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c9ed95c24ab44a9fa9ffe67c87bdd2fe: Generated new TSK 0
I20260807 15:15:59.666282 30758 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 15:15:59.671371 30472 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7fZSIL/build/release/bin/kudu with pid 30637
2026-08-07T15:15:59Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1444 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786115480851507-30472-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 15:15:59 dist-test-slave-42z9 krb5kdc[30792](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 15:15:59 dist-test-slave-42z9 krb5kdc[30792](info): set up 2 sockets
Aug 07 15:15:59 dist-test-slave-42z9 krb5kdc[30792](info): commencing operation
krb5kdc: starting...
W20260807 15:16:00.828593 30472 mini_kdc.cc:121] Time spent starting KDC: real 1.126s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786115480851507-30472-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786115480851507-30472-0/krb5kdc/test-user.keytab.
Aug 07 15:16:00 dist-test-slave-42z9 krb5kdc[30792](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115760, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T15:16:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T15:16:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.194.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.194.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.194.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.62.keytab.
Entry for principal kudu/127.29.194.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.62.keytab.
I20260807 15:16:00.968931 30472 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7fZSIL/build/release/bin/kudu
/tmp/dist-test-task7fZSIL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786115480851507-30472-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786115480851507-30472-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786115480851507-30472-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786115480851507-30472-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.194.62:45579
--webserver_interface=127.29.194.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.62.keytab
--principal=kudu/127.29.194.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.194.20:34763
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.194.62:45579
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786115480851507-30472-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786115480851507-30472-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786115480851507-30472-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:16:01.053740 30811 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:16:01.053958 30811 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:16:01.053990 30811 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:16:01.055116 30811 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:16:01.055163 30811 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:16:01.055182 30811 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:16:01.055199 30811 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:16:01.055215 30811 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:16:01.056697 30811 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.194.20:34763
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786115480851507-30472-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786115480851507-30472-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.194.62:45579
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.62.keytab
--principal=kudu/127.29.194.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.194.62:45579
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786115480851507-30472-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.194.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30811
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786115480851507-30472-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:16:01.056948 30811 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:16:01.057163 30811 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:16:01.062381 30818 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:16:01.062485 30811 server_base.cc:1034] running on GCE node
W20260807 15:16:01.062381 30821 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:16:01.062436 30819 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:16:01.063058 30811 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:16:01.063378 30811 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:16:01.064543 30811 hybrid_clock.cc:648] HybridClock initialized: now 1786115761064523 us; error 41 us; skew 500 ppm
Aug 07 15:16:01 dist-test-slave-42z9 krb5kdc[30792](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115761, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.194.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:16:01.072422 30811 init.cc:377] Logged in from keytab as kudu/127.29.194.62@KRBTEST.COM (short username kudu)
I20260807 15:16:01.072894 30811 webserver.cc:460] Webserver started at http://127.29.194.62:44281/ using document root <none> and password file <none>
I20260807 15:16:01.073117 30811 fs_manager.cc:359] Metadata directory not provided
I20260807 15:16:01.073166 30811 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:16:01.073302 30811 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:16:01.074460 30811 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786115480851507-30472-0/minicluster-data/master-0/data/instance:
uuid: "0cdecdf21fb8448b98c08552030b08b0"
format_stamp: "Formatted at 2026-08-07 15:16:01 on dist-test-slave-42z9"
server_key: "3ac855a5c8ff3db1f78829286f5ecfd6"
server_key_iv: "5f2dfe8699cf823ae4337d10a76fe12a"
server_key_version: "encryptionkey@0"
I20260807 15:16:01.074810 30811 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786115480851507-30472-0/minicluster-data/master-0/wal/instance:
uuid: "0cdecdf21fb8448b98c08552030b08b0"
format_stamp: "Formatted at 2026-08-07 15:16:01 on dist-test-slave-42z9"
server_key: "3ac855a5c8ff3db1f78829286f5ecfd6"
server_key_iv: "5f2dfe8699cf823ae4337d10a76fe12a"
server_key_version: "encryptionkey@0"
I20260807 15:16:01.076207 30811 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 15:16:01.076982 30835 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:16:01.077139 30811 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:16:01.077231 30811 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786115480851507-30472-0/minicluster-data/master-0/data,/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786115480851507-30472-0/minicluster-data/master-0/wal
uuid: "0cdecdf21fb8448b98c08552030b08b0"
format_stamp: "Formatted at 2026-08-07 15:16:01 on dist-test-slave-42z9"
server_key: "3ac855a5c8ff3db1f78829286f5ecfd6"
server_key_iv: "5f2dfe8699cf823ae4337d10a76fe12a"
server_key_version: "encryptionkey@0"
I20260807 15:16:01.077306 30811 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786115480851507-30472-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786115480851507-30472-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786115480851507-30472-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:16:01.103657 30811 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:16:01.106511 30811 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:16:01.106659 30811 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:16:01.110785 30811 rpc_server.cc:307] RPC server started. Bound to: 127.29.194.62:45579
I20260807 15:16:01.111255 30811 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786115480851507-30472-0/minicluster-data/master-0/data/info.pb
I20260807 15:16:01.111907 30908 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.194.62:45579 every 8 connection(s)
I20260807 15:16:01.112071 30911 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:16:01.114545 30911 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0cdecdf21fb8448b98c08552030b08b0: Bootstrap starting.
I20260807 15:16:01.115160 30911 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0cdecdf21fb8448b98c08552030b08b0: Neither blocks nor log segments found. Creating new log.
I20260807 15:16:01.115445 30911 log.cc:824] T 00000000000000000000000000000000 P 0cdecdf21fb8448b98c08552030b08b0: Log is configured to *not* fsync() on all Append() calls
I20260807 15:16:01.115979 30472 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7fZSIL/build/release/bin/kudu as pid 30811
I20260807 15:16:01.116088 30472 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786115480851507-30472-0/minicluster-data/master-0/wal/instance
I20260807 15:16:01.116206 30911 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0cdecdf21fb8448b98c08552030b08b0: No bootstrap required, opened a new log
I20260807 15:16:01.116279 30472 external_mini_cluster.cc:1428] Setting key 10e27f8fe2d5179bdda203024574e5fc
I20260807 15:16:01.119169 30911 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0cdecdf21fb8448b98c08552030b08b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0cdecdf21fb8448b98c08552030b08b0" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 45579 } }
I20260807 15:16:01.119371 30911 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0cdecdf21fb8448b98c08552030b08b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:16:01.119414 30911 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0cdecdf21fb8448b98c08552030b08b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0cdecdf21fb8448b98c08552030b08b0, State: Initialized, Role: FOLLOWER
I20260807 15:16:01.119546 30911 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0cdecdf21fb8448b98c08552030b08b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0cdecdf21fb8448b98c08552030b08b0" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 45579 } }
I20260807 15:16:01.119632 30911 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0cdecdf21fb8448b98c08552030b08b0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:16:01.119661 30911 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0cdecdf21fb8448b98c08552030b08b0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:16:01.119709 30911 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0cdecdf21fb8448b98c08552030b08b0 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:16:01.120316 30911 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0cdecdf21fb8448b98c08552030b08b0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0cdecdf21fb8448b98c08552030b08b0" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 45579 } }
I20260807 15:16:01.120433 30911 leader_election.cc:302] T 00000000000000000000000000000000 P 0cdecdf21fb8448b98c08552030b08b0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0cdecdf21fb8448b98c08552030b08b0; no voters:
I20260807 15:16:01.120597 30911 leader_election.cc:288] T 00000000000000000000000000000000 P 0cdecdf21fb8448b98c08552030b08b0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 15:16:01.120649 30917 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0cdecdf21fb8448b98c08552030b08b0 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:16:01.120850 30911 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0cdecdf21fb8448b98c08552030b08b0 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:16:01.120852 30917 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0cdecdf21fb8448b98c08552030b08b0 [term 1 LEADER]: Becoming Leader. State: Replica: 0cdecdf21fb8448b98c08552030b08b0, State: Running, Role: LEADER
I20260807 15:16:01.121086 30917 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0cdecdf21fb8448b98c08552030b08b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0cdecdf21fb8448b98c08552030b08b0" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 45579 } }
I20260807 15:16:01.121619 30919 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0cdecdf21fb8448b98c08552030b08b0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0cdecdf21fb8448b98c08552030b08b0. Latest consensus state: current_term: 1 leader_uuid: "0cdecdf21fb8448b98c08552030b08b0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0cdecdf21fb8448b98c08552030b08b0" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 45579 } } }
I20260807 15:16:01.121781 30919 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0cdecdf21fb8448b98c08552030b08b0 [sys.catalog]: This master's current role is: LEADER
I20260807 15:16:01.122040 30932 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:16:01.121635 30918 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0cdecdf21fb8448b98c08552030b08b0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0cdecdf21fb8448b98c08552030b08b0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0cdecdf21fb8448b98c08552030b08b0" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 45579 } } }
I20260807 15:16:01.122443 30918 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0cdecdf21fb8448b98c08552030b08b0 [sys.catalog]: This master's current role is: LEADER
I20260807 15:16:01.122824 30916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:16:01.116786 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44681 (local address 127.29.194.62:45579)
0807 15:16:01.117002 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0807 15:16:01.117011 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 15:16:01.117044 (+ 33us) server_negotiation.cc:407] Connection header received
0807 15:16:01.117639 (+ 595us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:16:01.117650 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:16:01.117713 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:16:01.117868 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:16:01.118111 (+ 243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:16:01.119482 (+ 1371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:16:01.120328 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:16:01.120964 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:16:01.121074 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:16:01.121846 (+ 772us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:16:01.121851 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:16:01.121851 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:16:01.122053 (+ 202us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:16:01.122094 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:16:01.122108 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:16:01.122167 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0807 15:16:01.122555 (+ 388us) server_negotiation.cc:299] Negotiation successful
0807 15:16:01.122627 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":157,"thread_start_us":110,"threads_started":1}
I20260807 15:16:01.122852 30932 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:16:01.123740 30472 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 15:16:01.124377 30932 catalog_manager.cc:1347] Generated new cluster ID: 8fd12bf16daf42dd9f920d5efdbfdd6a
I20260807 15:16:01.124423 30932 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:16:01.133179 30916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:16:01.125259 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42435 (local address 127.29.194.62:45579)
0807 15:16:01.125296 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0807 15:16:01.125298 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:16:01.129809 (+ 4511us) server_negotiation.cc:407] Connection header received
0807 15:16:01.129893 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:16:01.129894 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:16:01.129908 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:16:01.129958 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:16:01.130124 (+ 166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:16:01.130860 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:16:01.131869 (+ 1009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:16:01.132162 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:16:01.132221 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:16:01.132934 (+ 713us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:16:01.132935 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:16:01.132936 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:16:01.132941 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:16:01.132958 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:16:01.132962 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:16:01.133009 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0807 15:16:01.133065 (+ 56us) server_negotiation.cc:299] Negotiation successful
0807 15:16:01.133091 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260807 15:16:01.139559 30932 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:16:01.140322 30932 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:16:01.143108 30932 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0cdecdf21fb8448b98c08552030b08b0: Generated new TSK 0
I20260807 15:16:01.143301 30932 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 15:16:01.150539 30472 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7fZSIL/build/release/bin/kudu with pid 30811
2026-08-07T15:16:01Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1472 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786115480851507-30472-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 15:16:01 dist-test-slave-42z9 krb5kdc[30957](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 15:16:01 dist-test-slave-42z9 krb5kdc[30957](info): set up 2 sockets
Aug 07 15:16:01 dist-test-slave-42z9 krb5kdc[30957](info): commencing operation
krb5kdc: starting...
W20260807 15:16:02.584218 30472 mini_kdc.cc:121] Time spent starting KDC: real 1.409s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786115480851507-30472-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786115480851507-30472-0/krb5kdc/test-user.keytab.
Aug 07 15:16:02 dist-test-slave-42z9 krb5kdc[30957](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115762, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T15:16:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T15:16:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.194.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.194.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.194.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.62.keytab.
Entry for principal kudu/127.29.194.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.62.keytab.
I20260807 15:16:02.822603 30472 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7fZSIL/build/release/bin/kudu
/tmp/dist-test-task7fZSIL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786115480851507-30472-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786115480851507-30472-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786115480851507-30472-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786115480851507-30472-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.194.62:39315
--webserver_interface=127.29.194.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.62.keytab
--principal=kudu/127.29.194.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.194.20:46535
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.194.62:39315
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786115480851507-30472-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786115480851507-30472-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786115480851507-30472-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:16:02.969349 30982 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:16:02.969806 30982 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:16:02.969836 30982 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:16:02.971344 30982 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:16:02.971424 30982 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:16:02.971443 30982 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:16:02.971463 30982 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:16:02.971477 30982 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:16:02.973704 30982 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.194.20:46535
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786115480851507-30472-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786115480851507-30472-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.194.62:39315
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.62.keytab
--principal=kudu/127.29.194.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.194.62:39315
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786115480851507-30472-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.194.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30982
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786115480851507-30472-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:16:02.974092 30982 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:16:02.974555 30982 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:16:02.983006 30992 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:16:02.982983 30989 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:16:02.983033 30990 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:16:02.983412 30982 server_base.cc:1034] running on GCE node
I20260807 15:16:02.983788 30982 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:16:02.984349 30982 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:16:02.985533 30982 hybrid_clock.cc:648] HybridClock initialized: now 1786115762985504 us; error 52 us; skew 500 ppm
Aug 07 15:16:02 dist-test-slave-42z9 krb5kdc[30957](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115762, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.194.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:16:02.989594 30982 init.cc:377] Logged in from keytab as kudu/127.29.194.62@KRBTEST.COM (short username kudu)
I20260807 15:16:02.990936 30982 webserver.cc:460] Webserver started at http://127.29.194.62:35947/ using document root <none> and password file <none>
I20260807 15:16:02.991469 30982 fs_manager.cc:359] Metadata directory not provided
I20260807 15:16:02.991559 30982 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:16:02.991765 30982 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:16:02.993489 30982 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786115480851507-30472-0/minicluster-data/master-0/data/instance:
uuid: "db559e79a94b47e38832334e665fcc63"
format_stamp: "Formatted at 2026-08-07 15:16:02 on dist-test-slave-42z9"
server_key: "fbaca9859c84c800539149b752c7c7e8"
server_key_iv: "488da1274261afbca4f289adae234882"
server_key_version: "encryptionkey@0"
I20260807 15:16:02.994159 30982 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786115480851507-30472-0/minicluster-data/master-0/wal/instance:
uuid: "db559e79a94b47e38832334e665fcc63"
format_stamp: "Formatted at 2026-08-07 15:16:02 on dist-test-slave-42z9"
server_key: "fbaca9859c84c800539149b752c7c7e8"
server_key_iv: "488da1274261afbca4f289adae234882"
server_key_version: "encryptionkey@0"
I20260807 15:16:02.996676 30982 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.001s sys 0.002s
I20260807 15:16:02.998128 31004 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:16:02.998581 30982 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 15:16:02.998733 30982 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786115480851507-30472-0/minicluster-data/master-0/data,/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786115480851507-30472-0/minicluster-data/master-0/wal
uuid: "db559e79a94b47e38832334e665fcc63"
format_stamp: "Formatted at 2026-08-07 15:16:02 on dist-test-slave-42z9"
server_key: "fbaca9859c84c800539149b752c7c7e8"
server_key_iv: "488da1274261afbca4f289adae234882"
server_key_version: "encryptionkey@0"
I20260807 15:16:02.998921 30982 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786115480851507-30472-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786115480851507-30472-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786115480851507-30472-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:16:03.005290 30982 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:16:03.009716 30982 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:16:03.009966 30982 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:16:03.017303 30982 rpc_server.cc:307] RPC server started. Bound to: 127.29.194.62:39315
I20260807 15:16:03.017382 31076 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.194.62:39315 every 8 connection(s)
I20260807 15:16:03.017947 30982 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786115480851507-30472-0/minicluster-data/master-0/data/info.pb
I20260807 15:16:03.019516 31077 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:16:03.023430 30472 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7fZSIL/build/release/bin/kudu as pid 30982
I20260807 15:16:03.023619 30472 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786115480851507-30472-0/minicluster-data/master-0/wal/instance
I20260807 15:16:03.023970 30472 external_mini_cluster.cc:1428] Setting key d18683afb6aee22a79bb639d78ededc2
I20260807 15:16:03.024283 31077 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P db559e79a94b47e38832334e665fcc63: Bootstrap starting.
I20260807 15:16:03.025586 31077 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P db559e79a94b47e38832334e665fcc63: Neither blocks nor log segments found. Creating new log.
I20260807 15:16:03.026101 31077 log.cc:824] T 00000000000000000000000000000000 P db559e79a94b47e38832334e665fcc63: Log is configured to *not* fsync() on all Append() calls
I20260807 15:16:03.027525 31077 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P db559e79a94b47e38832334e665fcc63: No bootstrap required, opened a new log
I20260807 15:16:03.028680 31080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:16:03.024898 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58393 (local address 127.29.194.62:39315)
0807 15:16:03.025172 (+ 274us) server_negotiation.cc:206] Beginning negotiation
0807 15:16:03.025180 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 15:16:03.025228 (+ 48us) server_negotiation.cc:407] Connection header received
0807 15:16:03.026130 (+ 902us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:16:03.026146 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:16:03.026205 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:16:03.026457 (+ 252us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:16:03.027799 (+ 1342us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:16:03.027804 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:16:03.027804 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:16:03.027810 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:16:03.027895 (+ 85us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:16:03.027924 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:16:03.028083 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0807 15:16:03.028164 (+ 81us) server_negotiation.cc:299] Negotiation successful
0807 15:16:03.028255 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":184,"thread_start_us":120,"threads_started":1}
I20260807 15:16:03.031124 30472 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 15:16:03.031512 31077 raft_consensus.cc:348] T 00000000000000000000000000000000 P db559e79a94b47e38832334e665fcc63 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db559e79a94b47e38832334e665fcc63" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 39315 } }
I20260807 15:16:03.031791 31077 raft_consensus.cc:374] T 00000000000000000000000000000000 P db559e79a94b47e38832334e665fcc63 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:16:03.031839 31077 raft_consensus.cc:729] T 00000000000000000000000000000000 P db559e79a94b47e38832334e665fcc63 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: db559e79a94b47e38832334e665fcc63, State: Initialized, Role: FOLLOWER
I20260807 15:16:03.032037 31077 consensus_queue.cc:260] T 00000000000000000000000000000000 P db559e79a94b47e38832334e665fcc63 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db559e79a94b47e38832334e665fcc63" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 39315 } }
I20260807 15:16:03.032152 31077 raft_consensus.cc:388] T 00000000000000000000000000000000 P db559e79a94b47e38832334e665fcc63 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:16:03.032207 31077 raft_consensus.cc:482] T 00000000000000000000000000000000 P db559e79a94b47e38832334e665fcc63 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:16:03.032292 31077 raft_consensus.cc:3037] T 00000000000000000000000000000000 P db559e79a94b47e38832334e665fcc63 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:16:03.033478 31077 raft_consensus.cc:504] T 00000000000000000000000000000000 P db559e79a94b47e38832334e665fcc63 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db559e79a94b47e38832334e665fcc63" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 39315 } }
I20260807 15:16:03.033769 31077 leader_election.cc:302] T 00000000000000000000000000000000 P db559e79a94b47e38832334e665fcc63 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: db559e79a94b47e38832334e665fcc63; no voters:
I20260807 15:16:03.034098 31077 leader_election.cc:288] T 00000000000000000000000000000000 P db559e79a94b47e38832334e665fcc63 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 15:16:03.034173 31092 raft_consensus.cc:2781] T 00000000000000000000000000000000 P db559e79a94b47e38832334e665fcc63 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:16:03.034611 31077 sys_catalog.cc:564] T 00000000000000000000000000000000 P db559e79a94b47e38832334e665fcc63 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:16:03.034755 31092 raft_consensus.cc:686] T 00000000000000000000000000000000 P db559e79a94b47e38832334e665fcc63 [term 1 LEADER]: Becoming Leader. State: Replica: db559e79a94b47e38832334e665fcc63, State: Running, Role: LEADER
I20260807 15:16:03.035037 31092 consensus_queue.cc:237] T 00000000000000000000000000000000 P db559e79a94b47e38832334e665fcc63 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db559e79a94b47e38832334e665fcc63" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 39315 } }
I20260807 15:16:03.036036 31097 sys_catalog.cc:455] T 00000000000000000000000000000000 P db559e79a94b47e38832334e665fcc63 [sys.catalog]: SysCatalogTable state changed. Reason: New leader db559e79a94b47e38832334e665fcc63. Latest consensus state: current_term: 1 leader_uuid: "db559e79a94b47e38832334e665fcc63" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db559e79a94b47e38832334e665fcc63" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 39315 } } }
I20260807 15:16:03.036368 31093 sys_catalog.cc:455] T 00000000000000000000000000000000 P db559e79a94b47e38832334e665fcc63 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "db559e79a94b47e38832334e665fcc63" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db559e79a94b47e38832334e665fcc63" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 39315 } } }
I20260807 15:16:03.036540 31093 sys_catalog.cc:458] T 00000000000000000000000000000000 P db559e79a94b47e38832334e665fcc63 [sys.catalog]: This master's current role is: LEADER
I20260807 15:16:03.036875 31112 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:16:03.036330 31097 sys_catalog.cc:458] T 00000000000000000000000000000000 P db559e79a94b47e38832334e665fcc63 [sys.catalog]: This master's current role is: LEADER
I20260807 15:16:03.037393 31080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:16:03.033649 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33637 (local address 127.29.194.62:39315)
0807 15:16:03.033779 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0807 15:16:03.033782 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:16:03.033891 (+ 109us) server_negotiation.cc:407] Connection header received
0807 15:16:03.034080 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:16:03.034081 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:16:03.034087 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:16:03.034190 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:16:03.035891 (+ 1701us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:16:03.035892 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:16:03.035893 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:16:03.035899 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:16:03.035925 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:16:03.035940 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:16:03.036019 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0807 15:16:03.036276 (+ 257us) server_negotiation.cc:299] Negotiation successful
0807 15:16:03.036302 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91}
I20260807 15:16:03.038179 31112 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:16:03.040776 31112 catalog_manager.cc:1347] Generated new cluster ID: 856be2c059214491801ab84ead2df1cd
I20260807 15:16:03.040863 31112 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:16:03.094780 31112 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:16:03.095085 31112 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:16:03.105801 31112 catalog_manager.cc:5868] T 00000000000000000000000000000000 P db559e79a94b47e38832334e665fcc63: Generated new TSK 0
I20260807 15:16:03.106194 31112 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 15:16:03.114881 30472 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7fZSIL/build/release/bin/kudu with pid 30982
2026-08-07T15:16:03Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1973 ms)
[----------] 18 tests from AuthTokenIssuingTest (26784 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115480851507-30472-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 15:16:03 dist-test-slave-42z9 krb5kdc[31126](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 15:16:03 dist-test-slave-42z9 krb5kdc[31126](info): set up 2 sockets
Aug 07 15:16:03 dist-test-slave-42z9 krb5kdc[31126](info): commencing operation
krb5kdc: starting...
W20260807 15:16:04.343740 30472 mini_kdc.cc:121] Time spent starting KDC: real 1.195s user 0.000s sys 0.011s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115480851507-30472-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115480851507-30472-0/krb5kdc/test-user.keytab.
Aug 07 15:16:04 dist-test-slave-42z9 krb5kdc[31126](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115764, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T15:16:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T15:16:04Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.194.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.194.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.194.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.62.keytab.
Entry for principal kudu/127.29.194.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.62.keytab.
I20260807 15:16:04.481326 30472 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7fZSIL/build/release/bin/kudu
/tmp/dist-test-task7fZSIL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115480851507-30472-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115480851507-30472-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115480851507-30472-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115480851507-30472-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.194.62:40791
--webserver_interface=127.29.194.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.62.keytab
--principal=kudu/127.29.194.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.194.20:43209
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.194.62:40791,127.29.194.61:37549,127.29.194.60:39073
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115480851507-30472-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115480851507-30472-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115480851507-30472-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:16:04.567670 31142 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260807 15:16:04.567926 31142 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:16:04.567946 31142 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:16:04.567977 31142 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:16:04.569311 31142 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:16:04.569373 31142 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:16:04.569387 31142 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:16:04.569398 31142 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:16:04.569410 31142 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:16:04.571205 31142 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.194.20:43209
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115480851507-30472-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115480851507-30472-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.194.62:40791,127.29.194.61:37549,127.29.194.60:39073
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.62.keytab
--principal=kudu/127.29.194.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.194.62:40791
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115480851507-30472-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.194.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31142
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115480851507-30472-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:16:04.571498 31142 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:16:04.571755 31142 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:16:04.577044 31148 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:16:04.577011 31151 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:16:04.577016 31154 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:16:04.577055 31142 server_base.cc:1034] running on GCE node
I20260807 15:16:04.577603 31142 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:16:04.577952 31142 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:16:04.579120 31142 hybrid_clock.cc:648] HybridClock initialized: now 1786115764579098 us; error 39 us; skew 500 ppm
Aug 07 15:16:04 dist-test-slave-42z9 krb5kdc[31126](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115764, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.194.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:16:04.581828 31142 init.cc:377] Logged in from keytab as kudu/127.29.194.62@KRBTEST.COM (short username kudu)
I20260807 15:16:04.582302 31142 webserver.cc:460] Webserver started at http://127.29.194.62:36611/ using document root <none> and password file <none>
I20260807 15:16:04.582551 31142 fs_manager.cc:359] Metadata directory not provided
I20260807 15:16:04.582602 31142 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:16:04.582732 31142 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:16:04.583811 31142 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115480851507-30472-0/minicluster-data/master-0/data/instance:
uuid: "0c86b028a2c541be8a44a94a8371b06b"
format_stamp: "Formatted at 2026-08-07 15:16:04 on dist-test-slave-42z9"
server_key: "e96a9ea0fa8f4bb52ec31b8102322224"
server_key_iv: "bca2f77428102b085b98daa7aea391e2"
server_key_version: "encryptionkey@0"
I20260807 15:16:04.584144 31142 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115480851507-30472-0/minicluster-data/master-0/wal/instance:
uuid: "0c86b028a2c541be8a44a94a8371b06b"
format_stamp: "Formatted at 2026-08-07 15:16:04 on dist-test-slave-42z9"
server_key: "e96a9ea0fa8f4bb52ec31b8102322224"
server_key_iv: "bca2f77428102b085b98daa7aea391e2"
server_key_version: "encryptionkey@0"
I20260807 15:16:04.585566 31142 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 15:16:04.586409 31164 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:16:04.586803 31142 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:16:04.586921 31142 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115480851507-30472-0/minicluster-data/master-0/data,/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115480851507-30472-0/minicluster-data/master-0/wal
uuid: "0c86b028a2c541be8a44a94a8371b06b"
format_stamp: "Formatted at 2026-08-07 15:16:04 on dist-test-slave-42z9"
server_key: "e96a9ea0fa8f4bb52ec31b8102322224"
server_key_iv: "bca2f77428102b085b98daa7aea391e2"
server_key_version: "encryptionkey@0"
I20260807 15:16:04.586993 31142 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115480851507-30472-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115480851507-30472-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115480851507-30472-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:16:04.595737 31142 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:16:04.597829 31142 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:16:04.597980 31142 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:16:04.602435 31142 rpc_server.cc:307] RPC server started. Bound to: 127.29.194.62:40791
I20260807 15:16:04.602478 31234 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.194.62:40791 every 8 connection(s)
I20260807 15:16:04.602824 31142 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115480851507-30472-0/minicluster-data/master-0/data/info.pb
I20260807 15:16:04.603873 31236 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:16:04.605497 31236 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.194.62" port: 40791 } has no permanent_uuid. Determining permanent_uuid...
I20260807 15:16:04.608498 30472 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7fZSIL/build/release/bin/kudu as pid 31142
I20260807 15:16:04.608641 30472 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115480851507-30472-0/minicluster-data/master-0/wal/instance
I20260807 15:16:04.608829 30472 external_mini_cluster.cc:1428] Setting key c340b48ad0a5619f04e931ab2818080e
Aug 07 15:16:04 dist-test-slave-42z9 krb5kdc[31126](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115764, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.194.62@KRBTEST.COM for kudu/127.29.194.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.29.194.61@KRBTEST.COM; defaulting to no policy
I20260807 15:16:04.618165 31239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:16:04.606170 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:40791 (local address 127.0.0.1:47294)
0807 15:16:04.606544 (+ 374us) negotiation.cc:107] Waiting for socket to connect
0807 15:16:04.606604 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0807 15:16:04.607321 (+ 717us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:16:04.607762 (+ 441us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:16:04.607766 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:16:04.608147 (+ 381us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:16:04.608412 (+ 265us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:16:04.608417 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:16:04.610105 (+ 1688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:16:04.610106 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:16:04.611010 (+ 904us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:16:04.611013 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:16:04.611863 (+ 850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:16:04.611865 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:16:04.611953 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:16:04.612544 (+ 591us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:16:04.612553 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:16:04.614259 (+ 1706us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:16:04.616081 (+ 1822us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:16:04.616091 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:16:04.616094 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:16:04.616374 (+ 280us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:16:04.616856 (+ 482us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:16:04.616857 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:16:04.616857 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:16:04.616990 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:16:04.617561 (+ 571us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:16:04.617567 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:16:04.617639 (+ 72us) client_negotiation.cc:769] Sending connection context
0807 15:16:04.617735 (+ 96us) client_negotiation.cc:240] Negotiation successful
0807 15:16:04.617997 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":320,"mutex_wait_us":83,"thread_start_us":243,"threads_started":1}
I20260807 15:16:04.618165 31238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:16:04.606215 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47294 (local address 127.29.194.62:40791)
0807 15:16:04.606430 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0807 15:16:04.606438 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 15:16:04.606805 (+ 367us) server_negotiation.cc:407] Connection header received
0807 15:16:04.607553 (+ 748us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:16:04.607556 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:16:04.607582 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:16:04.607694 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:16:04.608554 (+ 860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:16:04.609944 (+ 1390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:16:04.611130 (+ 1186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:16:04.611675 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:16:04.611818 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:16:04.614384 (+ 2566us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:16:04.614393 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:16:04.614393 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:16:04.614400 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:16:04.615994 (+ 1594us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:16:04.616537 (+ 543us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:16:04.616543 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:16:04.616544 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:16:04.616737 (+ 193us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:16:04.617143 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:16:04.617145 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:16:04.617145 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:16:04.617434 (+ 289us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:16:04.617565 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0807 15:16:04.617772 (+ 207us) server_negotiation.cc:299] Negotiation successful
0807 15:16:04.617904 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":177,"thread_start_us":135,"threads_started":1}
I20260807 15:16:04.619339 31236 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.194.61" port: 37549 } has no permanent_uuid. Determining permanent_uuid...
I20260807 15:16:04.619729 31239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:16:04.619605 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.61:37549 (local address 127.0.0.1:35062)
0807 15:16:04.619654 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0807 15:16:04.619703 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.194.61:37549: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260807 15:16:04.619935 31170 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.29.194.61:37549: connect: Connection refused (error 111)
I20260807 15:16:04.620661 31239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:16:04.620509 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.61:37549 (local address 127.0.0.1:35070)
0807 15:16:04.620598 (+ 89us) negotiation.cc:107] Waiting for socket to connect
0807 15:16:04.620632 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.194.61:37549: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
W20260807 15:16:04.620846 31236 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.194.61:37549: Network error: Client connection negotiation failed: client connection to 127.29.194.61:37549: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.194.61@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.194.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.61.keytab.
Entry for principal kudu/127.29.194.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.61.keytab.
I20260807 15:16:04.638731 30472 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7fZSIL/build/release/bin/kudu
/tmp/dist-test-task7fZSIL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115480851507-30472-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115480851507-30472-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115480851507-30472-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115480851507-30472-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.194.61:37549
--webserver_interface=127.29.194.61
--webserver_port=0
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.61.keytab
--principal=kudu/127.29.194.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.194.20:43209
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.194.62:40791,127.29.194.61:37549,127.29.194.60:39073
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115480851507-30472-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115480851507-30472-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115480851507-30472-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 15:16:04.670107 31236 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.194.61" port: 37549 } attempt: 1
I20260807 15:16:04.670609 31239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:16:04.670484 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.61:37549 (local address 127.0.0.1:35086)
0807 15:16:04.670548 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0807 15:16:04.670582 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.194.61:37549: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260807 15:16:04.670974 31239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:16:04.670885 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.61:37549 (local address 127.0.0.1:35102)
0807 15:16:04.670930 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0807 15:16:04.670954 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.194.61:37549: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260807 15:16:04.671070 31236 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.194.61:37549: Network error: Client connection negotiation failed: client connection to 127.29.194.61:37549: connect: Connection refused (error 111)
W20260807 15:16:04.723951 31244 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260807 15:16:04.724200 31244 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:16:04.724224 31244 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:16:04.724252 31244 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:16:04.725503 31244 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:16:04.725551 31244 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:16:04.725569 31244 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:16:04.725585 31244 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:16:04.725601 31244 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:16:04.727272 31244 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.194.20:43209
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115480851507-30472-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115480851507-30472-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.29.194.62:40791,127.29.194.61:37549,127.29.194.60:39073
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.61.keytab
--principal=kudu/127.29.194.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.194.61:37549
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115480851507-30472-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.29.194.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31244
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115480851507-30472-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:16:04.727568 31244 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:16:04.727806 31244 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:16:04.732762 31253 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:16:04.732810 31250 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:16:04.732786 31251 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:16:04.732981 31244 server_base.cc:1034] running on GCE node
I20260807 15:16:04.733227 31244 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:16:04.733551 31244 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:16:04.734721 31244 hybrid_clock.cc:648] HybridClock initialized: now 1786115764734703 us; error 36 us; skew 500 ppm
Aug 07 15:16:04 dist-test-slave-42z9 krb5kdc[31126](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115764, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.194.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:16:04.737216 31244 init.cc:377] Logged in from keytab as kudu/127.29.194.61@KRBTEST.COM (short username kudu)
I20260807 15:16:04.737709 31244 webserver.cc:460] Webserver started at http://127.29.194.61:40897/ using document root <none> and password file <none>
I20260807 15:16:04.737924 31244 fs_manager.cc:359] Metadata directory not provided
I20260807 15:16:04.737963 31244 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:16:04.738056 31244 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:16:04.739056 31244 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115480851507-30472-0/minicluster-data/master-1/data/instance:
uuid: "e2778131dce245a6a60414bb2b7ddd5a"
format_stamp: "Formatted at 2026-08-07 15:16:04 on dist-test-slave-42z9"
server_key: "2c661b6658153b2a357f6dfe02b9d930"
server_key_iv: "83ca3b046c6718b29e6ca832e14a9402"
server_key_version: "encryptionkey@0"
I20260807 15:16:04.739244 31236 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.194.61" port: 37549 } attempt: 2
I20260807 15:16:04.739399 31244 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115480851507-30472-0/minicluster-data/master-1/wal/instance:
uuid: "e2778131dce245a6a60414bb2b7ddd5a"
format_stamp: "Formatted at 2026-08-07 15:16:04 on dist-test-slave-42z9"
server_key: "2c661b6658153b2a357f6dfe02b9d930"
server_key_iv: "83ca3b046c6718b29e6ca832e14a9402"
server_key_version: "encryptionkey@0"
I20260807 15:16:04.739652 31239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:16:04.739545 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.61:37549 (local address 127.0.0.1:35108)
0807 15:16:04.739589 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0807 15:16:04.739620 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.194.61:37549: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":20}
I20260807 15:16:04.739989 31239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:16:04.739904 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.61:37549 (local address 127.0.0.1:35124)
0807 15:16:04.739945 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0807 15:16:04.739968 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.194.61:37549: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260807 15:16:04.740072 31236 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.194.61:37549: Network error: Client connection negotiation failed: client connection to 127.29.194.61:37549: connect: Connection refused (error 111)
I20260807 15:16:04.740823 31244 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 15:16:04.741868 31261 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:16:04.742089 31244 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:16:04.742182 31244 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115480851507-30472-0/minicluster-data/master-1/data,/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115480851507-30472-0/minicluster-data/master-1/wal
uuid: "e2778131dce245a6a60414bb2b7ddd5a"
format_stamp: "Formatted at 2026-08-07 15:16:04 on dist-test-slave-42z9"
server_key: "2c661b6658153b2a357f6dfe02b9d930"
server_key_iv: "83ca3b046c6718b29e6ca832e14a9402"
server_key_version: "encryptionkey@0"
I20260807 15:16:04.742249 31244 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115480851507-30472-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115480851507-30472-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115480851507-30472-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:16:04.756834 31244 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:16:04.759035 31244 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:16:04.759181 31244 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:16:04.763397 31244 rpc_server.cc:307] RPC server started. Bound to: 127.29.194.61:37549
I20260807 15:16:04.763538 31337 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.194.61:37549 every 8 connection(s)
I20260807 15:16:04.763782 31244 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115480851507-30472-0/minicluster-data/master-1/data/info.pb
I20260807 15:16:04.764807 31338 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:16:04.765707 30472 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7fZSIL/build/release/bin/kudu as pid 31244
I20260807 15:16:04.765805 30472 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115480851507-30472-0/minicluster-data/master-1/wal/instance
I20260807 15:16:04.765982 30472 external_mini_cluster.cc:1428] Setting key 064c314c723f11001f5547d42893f31a
I20260807 15:16:04.766779 31338 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.194.62" port: 40791 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 15:16:04 dist-test-slave-42z9 krb5kdc[31126](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115764, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.194.61@KRBTEST.COM for kudu/127.29.194.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.29.194.60@KRBTEST.COM; defaulting to no policy
I20260807 15:16:04.776594 31238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:16:04.767631 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47302 (local address 127.29.194.62:40791)
0807 15:16:04.767689 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0807 15:16:04.767691 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:16:04.767904 (+ 213us) server_negotiation.cc:407] Connection header received
0807 15:16:04.768400 (+ 496us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:16:04.768402 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:16:04.768417 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:16:04.768473 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:16:04.769240 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:16:04.769786 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:16:04.771273 (+ 1487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:16:04.771487 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:16:04.771550 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:16:04.774065 (+ 2515us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:16:04.774067 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:16:04.774068 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:16:04.774076 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:16:04.775339 (+ 1263us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:16:04.775712 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:16:04.775713 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:16:04.775713 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:16:04.775764 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:16:04.775981 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:16:04.775981 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:16:04.775982 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:16:04.776094 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:16:04.776149 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0807 15:16:04.776408 (+ 259us) server_negotiation.cc:299] Negotiation successful
0807 15:16:04.776501 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260807 15:16:04.776746 31342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:16:04.767559 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:40791 (local address 127.0.0.1:47302)
0807 15:16:04.767797 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0807 15:16:04.767827 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0807 15:16:04.768306 (+ 479us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:16:04.768544 (+ 238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:16:04.768547 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:16:04.768837 (+ 290us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:16:04.769123 (+ 286us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:16:04.769129 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:16:04.769881 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:16:04.769882 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:16:04.771158 (+ 1276us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:16:04.771162 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:16:04.771585 (+ 423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:16:04.771586 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:16:04.771753 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:16:04.772285 (+ 532us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:16:04.772295 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:16:04.773965 (+ 1670us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:16:04.775418 (+ 1453us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:16:04.775427 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:16:04.775429 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:16:04.775646 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:16:04.775818 (+ 172us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:16:04.775818 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:16:04.775818 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:16:04.775924 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:16:04.776149 (+ 225us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:16:04.776157 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:16:04.776291 (+ 134us) client_negotiation.cc:769] Sending connection context
0807 15:16:04.776377 (+ 86us) client_negotiation.cc:240] Negotiation successful
0807 15:16:04.776536 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":206,"thread_start_us":94,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.194.60@KRBTEST.COM" created.
I20260807 15:16:04.777395 31338 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.194.61" port: 37549 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 15:16:04 dist-test-slave-42z9 krb5kdc[31126](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115764, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.194.61@KRBTEST.COM for kudu/127.29.194.61@KRBTEST.COM
I20260807 15:16:04.785079 31342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:16:04.777673 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.61:37549 (local address 127.0.0.1:35128)
0807 15:16:04.777718 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0807 15:16:04.777729 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 15:16:04.777772 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:16:04.778040 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:16:04.778041 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:16:04.778152 (+ 111us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:16:04.778200 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:16:04.778201 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:16:04.779218 (+ 1017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:16:04.779219 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:16:04.779967 (+ 748us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:16:04.779970 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:16:04.780602 (+ 632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:16:04.780603 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:16:04.780660 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:16:04.781031 (+ 371us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:16:04.781036 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:16:04.782221 (+ 1185us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:16:04.784118 (+ 1897us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:16:04.784120 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:16:04.784120 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:16:04.784287 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:16:04.784498 (+ 211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:16:04.784498 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:16:04.784498 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:16:04.784544 (+ 46us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:16:04.784853 (+ 309us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:16:04.784853 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:16:04.784898 (+ 45us) client_negotiation.cc:769] Sending connection context
0807 15:16:04.784944 (+ 46us) client_negotiation.cc:240] Negotiation successful
0807 15:16:04.785015 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":28}
I20260807 15:16:04.785202 31343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:16:04.777731 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35128 (local address 127.29.194.61:37549)
0807 15:16:04.777893 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0807 15:16:04.777898 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:16:04.777923 (+ 25us) server_negotiation.cc:407] Connection header received
0807 15:16:04.777945 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:16:04.777947 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:16:04.777960 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:16:04.778033 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:16:04.778263 (+ 230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:16:04.779124 (+ 861us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:16:04.780064 (+ 940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:16:04.780525 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:16:04.780596 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:16:04.782302 (+ 1706us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:16:04.782312 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:16:04.782312 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:16:04.782319 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:16:04.784019 (+ 1700us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:16:04.784373 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:16:04.784377 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:16:04.784377 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:16:04.784437 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:16:04.784597 (+ 160us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:16:04.784598 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:16:04.784598 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:16:04.784780 (+ 182us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:16:04.784881 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0807 15:16:04.784991 (+ 110us) server_negotiation.cc:299] Negotiation successful
0807 15:16:04.785071 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129,"thread_start_us":88,"threads_started":1}
I20260807 15:16:04.785813 31338 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.194.60" port: 39073 } has no permanent_uuid. Determining permanent_uuid...
I20260807 15:16:04.786151 31342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:16:04.786016 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.60:39073 (local address 127.0.0.1:48898)
0807 15:16:04.786079 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0807 15:16:04.786129 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.194.60:39073: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260807 15:16:04.786298 31262 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.29.194.60:39073: connect: Connection refused (error 111)
I20260807 15:16:04.786773 31342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:16:04.786681 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.60:39073 (local address 127.0.0.1:48906)
0807 15:16:04.786723 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0807 15:16:04.786753 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.194.60:39073: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
W20260807 15:16:04.786936 31338 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.194.60:39073: Network error: Client connection negotiation failed: client connection to 127.29.194.60:39073: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.194.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.60.keytab.
Entry for principal kudu/127.29.194.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.60.keytab.
I20260807 15:16:04.793046 30472 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7fZSIL/build/release/bin/kudu
/tmp/dist-test-task7fZSIL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115480851507-30472-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115480851507-30472-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115480851507-30472-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115480851507-30472-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.194.60:39073
--webserver_interface=127.29.194.60
--webserver_port=0
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.60.keytab
--principal=kudu/127.29.194.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.194.20:43209
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.194.62:40791,127.29.194.61:37549,127.29.194.60:39073
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115480851507-30472-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115480851507-30472-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115480851507-30472-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 15:16:04.831239 31236 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.194.61" port: 37549 } attempt: 3
Aug 07 15:16:04 dist-test-slave-42z9 krb5kdc[31126](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115764, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.194.62@KRBTEST.COM for kudu/127.29.194.61@KRBTEST.COM
I20260807 15:16:04.836203 31338 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.194.60" port: 39073 } attempt: 1
I20260807 15:16:04.836958 31342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:16:04.836753 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.60:39073 (local address 127.0.0.1:48922)
0807 15:16:04.836869 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0807 15:16:04.836919 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.194.60:39073: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":88}
I20260807 15:16:04.837482 31342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:16:04.837374 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.60:39073 (local address 127.0.0.1:48936)
0807 15:16:04.837408 (+ 34us) negotiation.cc:107] Waiting for socket to connect
0807 15:16:04.837458 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.194.60:39073: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":20}
W20260807 15:16:04.837630 31338 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.194.60:39073: Network error: Client connection negotiation failed: client connection to 127.29.194.60:39073: connect: Connection refused (error 111)
I20260807 15:16:04.839265 31343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:16:04.831820 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35134 (local address 127.29.194.61:37549)
0807 15:16:04.831896 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0807 15:16:04.831898 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:16:04.831933 (+ 35us) server_negotiation.cc:407] Connection header received
0807 15:16:04.831963 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:16:04.831964 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:16:04.831978 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:16:04.832026 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:16:04.832339 (+ 313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:16:04.833121 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:16:04.833873 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:16:04.834218 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:16:04.834288 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:16:04.836594 (+ 2306us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:16:04.836596 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:16:04.836596 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:16:04.836609 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:16:04.838223 (+ 1614us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:16:04.838622 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:16:04.838623 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:16:04.838623 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:16:04.838676 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:16:04.838843 (+ 167us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:16:04.838843 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:16:04.838843 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:16:04.838949 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:16:04.838992 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0807 15:16:04.839149 (+ 157us) server_negotiation.cc:299] Negotiation successful
0807 15:16:04.839210 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260807 15:16:04.839358 31239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:16:04.831759 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.61:37549 (local address 127.0.0.1:35134)
0807 15:16:04.831825 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0807 15:16:04.831838 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 15:16:04.831897 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:16:04.832045 (+ 148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:16:04.832046 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:16:04.832202 (+ 156us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:16:04.832271 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:16:04.832273 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:16:04.833202 (+ 929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:16:04.833203 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:16:04.833801 (+ 598us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:16:04.833803 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:16:04.834297 (+ 494us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:16:04.834298 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:16:04.834362 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:16:04.834929 (+ 567us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:16:04.834934 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:16:04.836435 (+ 1501us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:16:04.838310 (+ 1875us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:16:04.838311 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:16:04.838312 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:16:04.838553 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:16:04.838727 (+ 174us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:16:04.838728 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:16:04.838728 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:16:04.838791 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:16:04.838999 (+ 208us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:16:04.838999 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:16:04.839056 (+ 57us) client_negotiation.cc:769] Sending connection context
0807 15:16:04.839130 (+ 74us) client_negotiation.cc:240] Negotiation successful
0807 15:16:04.839246 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40}
I20260807 15:16:04.839815 31236 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.194.60" port: 39073 } has no permanent_uuid. Determining permanent_uuid...
I20260807 15:16:04.840144 31239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:16:04.840044 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.60:39073 (local address 127.0.0.1:48942)
0807 15:16:04.840086 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0807 15:16:04.840119 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.194.60:39073: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
I20260807 15:16:04.840476 31239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:16:04.840384 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.60:39073 (local address 127.0.0.1:48954)
0807 15:16:04.840432 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0807 15:16:04.840457 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.194.60:39073: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260807 15:16:04.840560 31236 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.194.60:39073: Network error: Client connection negotiation failed: client connection to 127.29.194.60:39073: connect: Connection refused (error 111)
I20260807 15:16:04.871747 31236 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.194.60" port: 39073 } attempt: 1
I20260807 15:16:04.872400 31239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:16:04.872171 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.60:39073 (local address 127.0.0.1:48962)
0807 15:16:04.872235 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0807 15:16:04.872335 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.194.60:39073: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260807 15:16:04.872869 31239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:16:04.872767 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.60:39073 (local address 127.0.0.1:48976)
0807 15:16:04.872804 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0807 15:16:04.872832 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.194.60:39073: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
W20260807 15:16:04.872961 31236 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.194.60:39073: Network error: Client connection negotiation failed: client connection to 127.29.194.60:39073: connect: Connection refused (error 111)
W20260807 15:16:04.880376 31347 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260807 15:16:04.880601 31347 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:16:04.880617 31347 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:16:04.880636 31347 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:16:04.881956 31347 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:16:04.882021 31347 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:16:04.882035 31347 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:16:04.882046 31347 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:16:04.882057 31347 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:16:04.883771 31347 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.194.20:43209
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115480851507-30472-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115480851507-30472-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.29.194.62:40791,127.29.194.61:37549,127.29.194.60:39073
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.60.keytab
--principal=kudu/127.29.194.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.194.60:39073
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115480851507-30472-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.29.194.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31347
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115480851507-30472-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:16:04.884035 31347 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:16:04.884277 31347 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:16:04.889297 31357 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:16:04.889298 31354 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:16:04.889396 31355 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:16:04.889655 31347 server_base.cc:1034] running on GCE node
I20260807 15:16:04.889936 31347 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:16:04.890229 31347 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:16:04.891386 31347 hybrid_clock.cc:648] HybridClock initialized: now 1786115764891371 us; error 32 us; skew 500 ppm
Aug 07 15:16:04 dist-test-slave-42z9 krb5kdc[31126](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115764, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.194.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:16:04.894562 31347 init.cc:377] Logged in from keytab as kudu/127.29.194.60@KRBTEST.COM (short username kudu)
I20260807 15:16:04.895064 31347 webserver.cc:460] Webserver started at http://127.29.194.60:46093/ using document root <none> and password file <none>
I20260807 15:16:04.895263 31347 fs_manager.cc:359] Metadata directory not provided
I20260807 15:16:04.895306 31347 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:16:04.895401 31347 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:16:04.896449 31347 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115480851507-30472-0/minicluster-data/master-2/data/instance:
uuid: "d4a332921c23475baf7aeee0e3677960"
format_stamp: "Formatted at 2026-08-07 15:16:04 on dist-test-slave-42z9"
server_key: "274c72740d01c7834c98a9bdd25e0769"
server_key_iv: "f198bbabe849b321fb6a049af2600a83"
server_key_version: "encryptionkey@0"
I20260807 15:16:04.896773 31347 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115480851507-30472-0/minicluster-data/master-2/wal/instance:
uuid: "d4a332921c23475baf7aeee0e3677960"
format_stamp: "Formatted at 2026-08-07 15:16:04 on dist-test-slave-42z9"
server_key: "274c72740d01c7834c98a9bdd25e0769"
server_key_iv: "f198bbabe849b321fb6a049af2600a83"
server_key_version: "encryptionkey@0"
I20260807 15:16:04.898116 31347 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260807 15:16:04.898996 31366 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:16:04.899351 31347 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260807 15:16:04.899447 31347 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115480851507-30472-0/minicluster-data/master-2/data,/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115480851507-30472-0/minicluster-data/master-2/wal
uuid: "d4a332921c23475baf7aeee0e3677960"
format_stamp: "Formatted at 2026-08-07 15:16:04 on dist-test-slave-42z9"
server_key: "274c72740d01c7834c98a9bdd25e0769"
server_key_iv: "f198bbabe849b321fb6a049af2600a83"
server_key_version: "encryptionkey@0"
I20260807 15:16:04.899511 31347 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115480851507-30472-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115480851507-30472-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115480851507-30472-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:16:04.905800 31338 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.194.60" port: 39073 } attempt: 2
I20260807 15:16:04.906293 31342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:16:04.906156 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.60:39073 (local address 127.0.0.1:48980)
0807 15:16:04.906214 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0807 15:16:04.906259 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.194.60:39073: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260807 15:16:04.906610 31342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:16:04.906514 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.60:39073 (local address 127.0.0.1:48988)
0807 15:16:04.906569 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0807 15:16:04.906593 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.194.60:39073: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260807 15:16:04.906692 31338 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.194.60:39073: Network error: Client connection negotiation failed: client connection to 127.29.194.60:39073: connect: Connection refused (error 111)
I20260807 15:16:04.909711 31347 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:16:04.911811 31347 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:16:04.911931 31347 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:16:04.915891 31347 rpc_server.cc:307] RPC server started. Bound to: 127.29.194.60:39073
I20260807 15:16:04.915933 31440 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.194.60:39073 every 8 connection(s)
I20260807 15:16:04.916275 31347 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115480851507-30472-0/minicluster-data/master-2/data/info.pb
I20260807 15:16:04.917817 31441 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:16:04.918689 30472 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7fZSIL/build/release/bin/kudu as pid 31347
I20260807 15:16:04.918790 30472 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115480851507-30472-0/minicluster-data/master-2/wal/instance
I20260807 15:16:04.918957 30472 external_mini_cluster.cc:1428] Setting key 0d66585e272beda966b28397f8742d43
I20260807 15:16:04.919746 31441 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.194.62" port: 40791 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 15:16:04 dist-test-slave-42z9 krb5kdc[31126](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115764, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.194.60@KRBTEST.COM for kudu/127.29.194.62@KRBTEST.COM
Aug 07 15:16:04 dist-test-slave-42z9 krb5kdc[31126](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115764, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.194.60@KRBTEST.COM
I20260807 15:16:04.929697 31238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:16:04.920917 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47316 (local address 127.29.194.62:40791)
0807 15:16:04.920970 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0807 15:16:04.920972 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:16:04.921098 (+ 126us) server_negotiation.cc:407] Connection header received
0807 15:16:04.921169 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:16:04.921170 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:16:04.921182 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:16:04.921243 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:16:04.921743 (+ 500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:16:04.922438 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:16:04.923696 (+ 1258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:16:04.923967 (+ 271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:16:04.924036 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:16:04.926458 (+ 2422us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:16:04.926460 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:16:04.926461 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:16:04.926470 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:16:04.928090 (+ 1620us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:16:04.928631 (+ 541us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:16:04.928632 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:16:04.928632 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:16:04.928702 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:16:04.928935 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:16:04.928936 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:16:04.928937 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:16:04.929072 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:16:04.929141 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0807 15:16:04.929508 (+ 367us) server_negotiation.cc:299] Negotiation successful
0807 15:16:04.929597 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260807 15:16:04.929921 31447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:16:04.919533 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48994 (local address 127.29.194.60:39073)
0807 15:16:04.919706 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0807 15:16:04.919714 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 15:16:04.919740 (+ 26us) server_negotiation.cc:407] Connection header received
0807 15:16:04.920272 (+ 532us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:16:04.920279 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:16:04.920327 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:16:04.920460 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:16:04.921200 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:16:04.922473 (+ 1273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:16:04.923547 (+ 1074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:16:04.923949 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:16:04.924072 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:16:04.926935 (+ 2863us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:16:04.926943 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:16:04.926943 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:16:04.926949 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:16:04.928588 (+ 1639us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:16:04.928938 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:16:04.928940 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:16:04.928941 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:16:04.928988 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:16:04.929201 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:16:04.929202 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:16:04.929203 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:16:04.929357 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:16:04.929518 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0807 15:16:04.929644 (+ 126us) server_negotiation.cc:299] Negotiation successful
0807 15:16:04.929722 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127,"thread_start_us":80,"threads_started":1}
I20260807 15:16:04.930055 31448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:16:04.920805 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:40791 (local address 127.0.0.1:47316)
0807 15:16:04.920975 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0807 15:16:04.921023 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0807 15:16:04.921107 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:16:04.921267 (+ 160us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:16:04.921269 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:16:04.921564 (+ 295us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:16:04.921660 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:16:04.921665 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:16:04.922579 (+ 914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:16:04.922579 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:16:04.923618 (+ 1039us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:16:04.923620 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:16:04.924073 (+ 453us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:16:04.924074 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:16:04.924151 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:16:04.924719 (+ 568us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:16:04.924729 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:16:04.926363 (+ 1634us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:16:04.928178 (+ 1815us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:16:04.928184 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:16:04.928186 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:16:04.928551 (+ 365us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:16:04.928769 (+ 218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:16:04.928770 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:16:04.928770 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:16:04.928873 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:16:04.929141 (+ 268us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:16:04.929147 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:16:04.929319 (+ 172us) client_negotiation.cc:769] Sending connection context
0807 15:16:04.929426 (+ 107us) client_negotiation.cc:240] Negotiation successful
0807 15:16:04.929634 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":88,"threads_started":1}
I20260807 15:16:04.930662 31441 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.194.61" port: 37549 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 15:16:04 dist-test-slave-42z9 krb5kdc[31126](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115764, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.194.60@KRBTEST.COM for kudu/127.29.194.61@KRBTEST.COM
Aug 07 15:16:04 dist-test-slave-42z9 krb5kdc[31126](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115764, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.194.62@KRBTEST.COM
I20260807 15:16:04.937867 31343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:16:04.930966 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35138 (local address 127.29.194.61:37549)
0807 15:16:04.931005 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0807 15:16:04.931007 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:16:04.931014 (+ 7us) server_negotiation.cc:407] Connection header received
0807 15:16:04.931037 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:16:04.931038 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:16:04.931048 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:16:04.931106 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:16:04.931322 (+ 216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:16:04.931860 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:16:04.932534 (+ 674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:16:04.932787 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:16:04.932845 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:16:04.934848 (+ 2003us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:16:04.934850 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:16:04.934851 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:16:04.934858 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:16:04.936103 (+ 1245us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:16:04.936699 (+ 596us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:16:04.936701 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:16:04.936702 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:16:04.936821 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:16:04.937080 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:16:04.937081 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:16:04.937082 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:16:04.937413 (+ 331us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:16:04.937522 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0807 15:16:04.937679 (+ 157us) server_negotiation.cc:299] Negotiation successful
0807 15:16:04.937766 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260807 15:16:04.937867 31448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:16:04.930898 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.61:37549 (local address 127.0.0.1:35138)
0807 15:16:04.930935 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0807 15:16:04.930943 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0807 15:16:04.930971 (+ 28us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:16:04.931108 (+ 137us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:16:04.931109 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:16:04.931218 (+ 109us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:16:04.931261 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:16:04.931262 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:16:04.931926 (+ 664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:16:04.931926 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:16:04.932459 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:16:04.932462 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:16:04.932850 (+ 388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:16:04.932851 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:16:04.932919 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:16:04.933388 (+ 469us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:16:04.933392 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:16:04.934761 (+ 1369us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:16:04.936180 (+ 1419us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:16:04.936181 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:16:04.936182 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:16:04.936529 (+ 347us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:16:04.936922 (+ 393us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:16:04.936923 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:16:04.936923 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:16:04.937005 (+ 82us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:16:04.937528 (+ 523us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:16:04.937529 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:16:04.937582 (+ 53us) client_negotiation.cc:769] Sending connection context
0807 15:16:04.937652 (+ 70us) client_negotiation.cc:240] Negotiation successful
0807 15:16:04.937767 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":25}
I20260807 15:16:04.938056 31238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:16:04.930746 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47326 (local address 127.29.194.62:40791)
0807 15:16:04.930783 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0807 15:16:04.930785 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:16:04.930834 (+ 49us) server_negotiation.cc:407] Connection header received
0807 15:16:04.930889 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:16:04.930890 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:16:04.930900 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:16:04.930953 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:16:04.931507 (+ 554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:16:04.931971 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:16:04.932605 (+ 634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:16:04.932778 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:16:04.932829 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:16:04.935371 (+ 2542us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:16:04.935373 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:16:04.935373 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:16:04.935380 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:16:04.936689 (+ 1309us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:16:04.937271 (+ 582us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:16:04.937272 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:16:04.937272 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:16:04.937320 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:16:04.937538 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:16:04.937539 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:16:04.937539 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:16:04.937672 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:16:04.937837 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0807 15:16:04.937887 (+ 50us) server_negotiation.cc:299] Negotiation successful
0807 15:16:04.937960 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260807 15:16:04.938795 31441 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.194.60" port: 39073 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 15:16:04 dist-test-slave-42z9 krb5kdc[31126](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115764, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.194.60@KRBTEST.COM for kudu/127.29.194.60@KRBTEST.COM
Aug 07 15:16:04 dist-test-slave-42z9 krb5kdc[31126](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115764, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.194.61@KRBTEST.COM
I20260807 15:16:04.945153 31448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:16:04.939053 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.60:39073 (local address 127.0.0.1:49008)
0807 15:16:04.939092 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0807 15:16:04.939101 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0807 15:16:04.939136 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:16:04.939230 (+ 94us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:16:04.939230 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:16:04.939373 (+ 143us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:16:04.939406 (+ 33us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:16:04.939408 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:16:04.940013 (+ 605us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:16:04.940014 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:16:04.940489 (+ 475us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:16:04.940490 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:16:04.940843 (+ 353us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:16:04.940843 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:16:04.940896 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:16:04.941329 (+ 433us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:16:04.941333 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:16:04.942560 (+ 1227us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:16:04.944084 (+ 1524us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:16:04.944086 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:16:04.944086 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:16:04.944299 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:16:04.944555 (+ 256us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:16:04.944556 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:16:04.944557 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:16:04.944624 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:16:04.944893 (+ 269us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:16:04.944894 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:16:04.944939 (+ 45us) client_negotiation.cc:769] Sending connection context
0807 15:16:04.944990 (+ 51us) client_negotiation.cc:240] Negotiation successful
0807 15:16:04.945082 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":23}
I20260807 15:16:04.945255 31447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:16:04.939103 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49008 (local address 127.29.194.60:39073)
0807 15:16:04.939137 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0807 15:16:04.939139 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:16:04.939146 (+ 7us) server_negotiation.cc:407] Connection header received
0807 15:16:04.939180 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:16:04.939180 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:16:04.939188 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:16:04.939228 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:16:04.939472 (+ 244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:16:04.939940 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:16:04.940564 (+ 624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:16:04.940781 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:16:04.940836 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:16:04.942637 (+ 1801us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:16:04.942639 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:16:04.942639 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:16:04.942646 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:16:04.944017 (+ 1371us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:16:04.944425 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:16:04.944426 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:16:04.944427 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:16:04.944487 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:16:04.944683 (+ 196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:16:04.944684 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:16:04.944684 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:16:04.944829 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:16:04.944892 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0807 15:16:04.945030 (+ 138us) server_negotiation.cc:299] Negotiation successful
0807 15:16:04.945139 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":30,"server-negotiator.queue_time_us":19}
I20260807 15:16:04.946236 31343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:16:04.938700 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35142 (local address 127.29.194.61:37549)
0807 15:16:04.938738 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0807 15:16:04.938739 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 15:16:04.938786 (+ 47us) server_negotiation.cc:407] Connection header received
0807 15:16:04.938857 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:16:04.938858 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:16:04.938867 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:16:04.938912 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:16:04.939426 (+ 514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:16:04.939911 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:16:04.940569 (+ 658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:16:04.940778 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:16:04.940833 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:16:04.943152 (+ 2319us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:16:04.943153 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:16:04.943154 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:16:04.943160 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:16:04.944823 (+ 1663us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:16:04.945396 (+ 573us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:16:04.945397 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:16:04.945397 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:16:04.945479 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:16:04.945658 (+ 179us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:16:04.945659 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:16:04.945660 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:16:04.945803 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:16:04.945970 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0807 15:16:04.946064 (+ 94us) server_negotiation.cc:299] Negotiation successful
0807 15:16:04.946156 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260807 15:16:04.946920 31441 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d4a332921c23475baf7aeee0e3677960: Bootstrap starting.
I20260807 15:16:04.947778 31441 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d4a332921c23475baf7aeee0e3677960: Neither blocks nor log segments found. Creating new log.
I20260807 15:16:04.948076 31441 log.cc:824] T 00000000000000000000000000000000 P d4a332921c23475baf7aeee0e3677960: Log is configured to *not* fsync() on all Append() calls
I20260807 15:16:04.948107 31236 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.194.60" port: 39073 } attempt: 2
I20260807 15:16:04.948942 31441 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d4a332921c23475baf7aeee0e3677960: No bootstrap required, opened a new log
I20260807 15:16:04.949761 30472 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 15:16:04.950964 31441 raft_consensus.cc:348] T 00000000000000000000000000000000 P d4a332921c23475baf7aeee0e3677960 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c86b028a2c541be8a44a94a8371b06b" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 40791 } } peers { permanent_uuid: "e2778131dce245a6a60414bb2b7ddd5a" member_type: VOTER last_known_addr { host: "127.29.194.61" port: 37549 } } peers { permanent_uuid: "d4a332921c23475baf7aeee0e3677960" member_type: VOTER last_known_addr { host: "127.29.194.60" port: 39073 } }
I20260807 15:16:04.951090 31441 raft_consensus.cc:729] T 00000000000000000000000000000000 P d4a332921c23475baf7aeee0e3677960 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d4a332921c23475baf7aeee0e3677960, State: Initialized, Role: FOLLOWER
I20260807 15:16:04.951165 31441 consensus_queue.cc:260] T 00000000000000000000000000000000 P d4a332921c23475baf7aeee0e3677960 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c86b028a2c541be8a44a94a8371b06b" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 40791 } } peers { permanent_uuid: "e2778131dce245a6a60414bb2b7ddd5a" member_type: VOTER last_known_addr { host: "127.29.194.61" port: 37549 } } peers { permanent_uuid: "d4a332921c23475baf7aeee0e3677960" member_type: VOTER last_known_addr { host: "127.29.194.60" port: 39073 } }
I20260807 15:16:04.951511 31441 sys_catalog.cc:564] T 00000000000000000000000000000000 P d4a332921c23475baf7aeee0e3677960 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:16:04.951473 31451 sys_catalog.cc:455] T 00000000000000000000000000000000 P d4a332921c23475baf7aeee0e3677960 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c86b028a2c541be8a44a94a8371b06b" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 40791 } } peers { permanent_uuid: "e2778131dce245a6a60414bb2b7ddd5a" member_type: VOTER last_known_addr { host: "127.29.194.61" port: 37549 } } peers { permanent_uuid: "d4a332921c23475baf7aeee0e3677960" member_type: VOTER last_known_addr { host: "127.29.194.60" port: 39073 } } }
I20260807 15:16:04.951349 31202 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "0c86b028a2c541be8a44a94a8371b06b"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47326
Aug 07 15:16:04 dist-test-slave-42z9 krb5kdc[31126](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115764, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.194.62@KRBTEST.COM for kudu/127.29.194.60@KRBTEST.COM
I20260807 15:16:04.951591 31451 sys_catalog.cc:458] T 00000000000000000000000000000000 P d4a332921c23475baf7aeee0e3677960 [sys.catalog]: This master's current role is: FOLLOWER
W20260807 15:16:04.953816 31469 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d4a332921c23475baf7aeee0e3677960: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 15:16:04.953891 31469 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 15:16:04.956291 31447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:16:04.948459 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49024 (local address 127.29.194.60:39073)
0807 15:16:04.948508 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0807 15:16:04.948510 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:16:04.948590 (+ 80us) server_negotiation.cc:407] Connection header received
0807 15:16:04.948727 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:16:04.948728 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:16:04.948737 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:16:04.948784 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:16:04.949111 (+ 327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:16:04.949809 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:16:04.950788 (+ 979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:16:04.951035 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:16:04.951116 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:16:04.953283 (+ 2167us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:16:04.953285 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:16:04.953285 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:16:04.953292 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:16:04.955064 (+ 1772us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:16:04.955463 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:16:04.955464 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:16:04.955464 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:16:04.955533 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:16:04.955729 (+ 196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:16:04.955730 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:16:04.955730 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:16:04.955902 (+ 172us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:16:04.955964 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0807 15:16:04.956121 (+ 157us) server_negotiation.cc:299] Negotiation successful
0807 15:16:04.956199 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260807 15:16:04.956292 31239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:16:04.948413 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.60:39073 (local address 127.0.0.1:49024)
0807 15:16:04.948537 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0807 15:16:04.948549 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 15:16:04.948601 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:16:04.948804 (+ 203us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:16:04.948805 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:16:04.948979 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:16:04.949028 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:16:04.949030 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:16:04.949887 (+ 857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:16:04.949889 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:16:04.950685 (+ 796us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:16:04.950688 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:16:04.951128 (+ 440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:16:04.951130 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:16:04.951191 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:16:04.951653 (+ 462us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:16:04.951658 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:16:04.953195 (+ 1537us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:16:04.955162 (+ 1967us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:16:04.955164 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:16:04.955164 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:16:04.955390 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:16:04.955607 (+ 217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:16:04.955608 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:16:04.955608 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:16:04.955669 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:16:04.955955 (+ 286us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:16:04.955956 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:16:04.956022 (+ 66us) client_negotiation.cc:769] Sending connection context
0807 15:16:04.956081 (+ 59us) client_negotiation.cc:240] Negotiation successful
0807 15:16:04.956199 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107}
I20260807 15:16:04.953842 31347 master_runner.cc:186] Error getting master registration for 127.29.194.62:40791: OK, instance_id { permanent_uuid: "0c86b028a2c541be8a44a94a8371b06b" instance_seqno: 1786115764601029 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260807 15:16:04.956861 31347 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260807 15:16:04.957893 31236 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0c86b028a2c541be8a44a94a8371b06b: Bootstrap starting.
I20260807 15:16:04.958669 31236 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0c86b028a2c541be8a44a94a8371b06b: Neither blocks nor log segments found. Creating new log.
I20260807 15:16:04.958968 31236 log.cc:824] T 00000000000000000000000000000000 P 0c86b028a2c541be8a44a94a8371b06b: Log is configured to *not* fsync() on all Append() calls
I20260807 15:16:04.959791 31236 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0c86b028a2c541be8a44a94a8371b06b: No bootstrap required, opened a new log
I20260807 15:16:04.960857 31236 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0c86b028a2c541be8a44a94a8371b06b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c86b028a2c541be8a44a94a8371b06b" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 40791 } } peers { permanent_uuid: "e2778131dce245a6a60414bb2b7ddd5a" member_type: VOTER last_known_addr { host: "127.29.194.61" port: 37549 } } peers { permanent_uuid: "d4a332921c23475baf7aeee0e3677960" member_type: VOTER last_known_addr { host: "127.29.194.60" port: 39073 } }
I20260807 15:16:04.960987 31236 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0c86b028a2c541be8a44a94a8371b06b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0c86b028a2c541be8a44a94a8371b06b, State: Initialized, Role: FOLLOWER
I20260807 15:16:04.961059 31236 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0c86b028a2c541be8a44a94a8371b06b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c86b028a2c541be8a44a94a8371b06b" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 40791 } } peers { permanent_uuid: "e2778131dce245a6a60414bb2b7ddd5a" member_type: VOTER last_known_addr { host: "127.29.194.61" port: 37549 } } peers { permanent_uuid: "d4a332921c23475baf7aeee0e3677960" member_type: VOTER last_known_addr { host: "127.29.194.60" port: 39073 } }
I20260807 15:16:04.961402 31236 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0c86b028a2c541be8a44a94a8371b06b [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:16:04.961360 31473 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0c86b028a2c541be8a44a94a8371b06b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c86b028a2c541be8a44a94a8371b06b" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 40791 } } peers { permanent_uuid: "e2778131dce245a6a60414bb2b7ddd5a" member_type: VOTER last_known_addr { host: "127.29.194.61" port: 37549 } } peers { permanent_uuid: "d4a332921c23475baf7aeee0e3677960" member_type: VOTER last_known_addr { host: "127.29.194.60" port: 39073 } } }
I20260807 15:16:04.961509 31202 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0c86b028a2c541be8a44a94a8371b06b [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260807 15:16:04.961573 31202 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0c86b028a2c541be8a44a94a8371b06b [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:16:04.962426 31202 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0c86b028a2c541be8a44a94a8371b06b [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c86b028a2c541be8a44a94a8371b06b" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 40791 } } peers { permanent_uuid: "e2778131dce245a6a60414bb2b7ddd5a" member_type: VOTER last_known_addr { host: "127.29.194.61" port: 37549 } } peers { permanent_uuid: "d4a332921c23475baf7aeee0e3677960" member_type: VOTER last_known_addr { host: "127.29.194.60" port: 39073 } }
I20260807 15:16:04.962702 31202 leader_election.cc:288] T 00000000000000000000000000000000 P 0c86b028a2c541be8a44a94a8371b06b [CANDIDATE]: Term 1 election: Requested vote from peers e2778131dce245a6a60414bb2b7ddd5a (127.29.194.61:37549), d4a332921c23475baf7aeee0e3677960 (127.29.194.60:39073)
W20260807 15:16:04.963531 31491 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 0c86b028a2c541be8a44a94a8371b06b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 15:16:04.963673 31491 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 15:16:04.964363 31400 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0c86b028a2c541be8a44a94a8371b06b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "d4a332921c23475baf7aeee0e3677960"
I20260807 15:16:04.964502 31400 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d4a332921c23475baf7aeee0e3677960 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:16:04.961504 31473 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0c86b028a2c541be8a44a94a8371b06b [sys.catalog]: This master's current role is: FOLLOWER
I20260807 15:16:04.965608 31400 raft_consensus.cc:2454] T 00000000000000000000000000000000 P d4a332921c23475baf7aeee0e3677960 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0c86b028a2c541be8a44a94a8371b06b in term 1.
I20260807 15:16:04.965883 31165 leader_election.cc:302] T 00000000000000000000000000000000 P 0c86b028a2c541be8a44a94a8371b06b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0c86b028a2c541be8a44a94a8371b06b, d4a332921c23475baf7aeee0e3677960; no voters:
I20260807 15:16:04.966014 31473 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0c86b028a2c541be8a44a94a8371b06b [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:16:04.966163 31473 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0c86b028a2c541be8a44a94a8371b06b [term 1 LEADER]: Becoming Leader. State: Replica: 0c86b028a2c541be8a44a94a8371b06b, State: Running, Role: LEADER
I20260807 15:16:04.966396 31473 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0c86b028a2c541be8a44a94a8371b06b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c86b028a2c541be8a44a94a8371b06b" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 40791 } } peers { permanent_uuid: "e2778131dce245a6a60414bb2b7ddd5a" member_type: VOTER last_known_addr { host: "127.29.194.61" port: 37549 } } peers { permanent_uuid: "d4a332921c23475baf7aeee0e3677960" member_type: VOTER last_known_addr { host: "127.29.194.60" port: 39073 } }
I20260807 15:16:04.967238 31504 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0c86b028a2c541be8a44a94a8371b06b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0c86b028a2c541be8a44a94a8371b06b. Latest consensus state: current_term: 1 leader_uuid: "0c86b028a2c541be8a44a94a8371b06b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c86b028a2c541be8a44a94a8371b06b" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 40791 } } peers { permanent_uuid: "e2778131dce245a6a60414bb2b7ddd5a" member_type: VOTER last_known_addr { host: "127.29.194.61" port: 37549 } } peers { permanent_uuid: "d4a332921c23475baf7aeee0e3677960" member_type: VOTER last_known_addr { host: "127.29.194.60" port: 39073 } } }
I20260807 15:16:04.967460 31504 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0c86b028a2c541be8a44a94a8371b06b [sys.catalog]: This master's current role is: LEADER
I20260807 15:16:04.965090 31142 master_runner.cc:186] Error getting master registration for 127.29.194.61:37549: OK, instance_id { permanent_uuid: "e2778131dce245a6a60414bb2b7ddd5a" instance_seqno: 1786115764762259 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260807 15:16:04.967602 31142 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260807 15:16:04.974732 31238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:16:04.965030 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47342 (local address 127.29.194.62:40791)
0807 15:16:04.965069 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0807 15:16:04.965071 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:16:04.966505 (+ 1434us) server_negotiation.cc:407] Connection header received
0807 15:16:04.966600 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:16:04.966602 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:16:04.966611 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:16:04.966637 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:16:04.967440 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:16:04.968156 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:16:04.969173 (+ 1017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:16:04.969509 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:16:04.969552 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:16:04.971163 (+ 1611us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:16:04.971165 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:16:04.971166 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:16:04.971174 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:16:04.972849 (+ 1675us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:16:04.973514 (+ 665us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:16:04.973528 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:16:04.973528 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:16:04.973592 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:16:04.973898 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:16:04.973899 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:16:04.973899 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:16:04.974050 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:16:04.974139 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0807 15:16:04.974538 (+ 399us) server_negotiation.cc:299] Negotiation successful
0807 15:16:04.974625 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260807 15:16:04.979215 31508 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:16:04.979687 31508 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:16:04.982511 31400 raft_consensus.cc:1261] T 00000000000000000000000000000000 P d4a332921c23475baf7aeee0e3677960 [term 1 FOLLOWER]: Refusing update from remote peer 0c86b028a2c541be8a44a94a8371b06b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:16:04.983035 31504 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 0c86b028a2c541be8a44a94a8371b06b [LEADER]: Connected to new peer: Peer: permanent_uuid: "d4a332921c23475baf7aeee0e3677960" member_type: VOTER last_known_addr { host: "127.29.194.60" port: 39073 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:16:04.984596 31447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:16:04.965721 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49026 (local address 127.29.194.60:39073)
0807 15:16:04.965794 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0807 15:16:04.965796 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:16:04.977595 (+ 11799us) server_negotiation.cc:407] Connection header received
0807 15:16:04.977696 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:16:04.977697 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:16:04.977709 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:16:04.977737 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:16:04.978449 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:16:04.979104 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:16:04.980052 (+ 948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:16:04.980323 (+ 271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:16:04.980365 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:16:04.981717 (+ 1352us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:16:04.981718 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:16:04.981718 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:16:04.981726 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:16:04.983346 (+ 1620us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:16:04.983767 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:16:04.983768 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:16:04.983768 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:16:04.983825 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:16:04.984014 (+ 189us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:16:04.984015 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:16:04.984015 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:16:04.984144 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:16:04.984210 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0807 15:16:04.984411 (+ 201us) server_negotiation.cc:299] Negotiation successful
0807 15:16:04.984504 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260807 15:16:04.985365 31451 sys_catalog.cc:455] T 00000000000000000000000000000000 P d4a332921c23475baf7aeee0e3677960 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0c86b028a2c541be8a44a94a8371b06b. Latest consensus state: current_term: 1 leader_uuid: "0c86b028a2c541be8a44a94a8371b06b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c86b028a2c541be8a44a94a8371b06b" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 40791 } } peers { permanent_uuid: "e2778131dce245a6a60414bb2b7ddd5a" member_type: VOTER last_known_addr { host: "127.29.194.61" port: 37549 } } peers { permanent_uuid: "d4a332921c23475baf7aeee0e3677960" member_type: VOTER last_known_addr { host: "127.29.194.60" port: 39073 } } }
I20260807 15:16:04.985476 31451 sys_catalog.cc:458] T 00000000000000000000000000000000 P d4a332921c23475baf7aeee0e3677960 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 15:16:04.987630 31504 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0c86b028a2c541be8a44a94a8371b06b [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0c86b028a2c541be8a44a94a8371b06b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c86b028a2c541be8a44a94a8371b06b" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 40791 } } peers { permanent_uuid: "e2778131dce245a6a60414bb2b7ddd5a" member_type: VOTER last_known_addr { host: "127.29.194.61" port: 37549 } } peers { permanent_uuid: "d4a332921c23475baf7aeee0e3677960" member_type: VOTER last_known_addr { host: "127.29.194.60" port: 39073 } } }
I20260807 15:16:04.987732 31504 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0c86b028a2c541be8a44a94a8371b06b [sys.catalog]: This master's current role is: LEADER
I20260807 15:16:04.988241 31451 sys_catalog.cc:455] T 00000000000000000000000000000000 P d4a332921c23475baf7aeee0e3677960 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0c86b028a2c541be8a44a94a8371b06b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c86b028a2c541be8a44a94a8371b06b" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 40791 } } peers { permanent_uuid: "e2778131dce245a6a60414bb2b7ddd5a" member_type: VOTER last_known_addr { host: "127.29.194.61" port: 37549 } } peers { permanent_uuid: "d4a332921c23475baf7aeee0e3677960" member_type: VOTER last_known_addr { host: "127.29.194.60" port: 39073 } } }
I20260807 15:16:04.988320 31508 catalog_manager.cc:1347] Generated new cluster ID: bf222aa5df8845a2a1cdc136cfcb4637
I20260807 15:16:04.988370 31508 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:16:04.988329 31451 sys_catalog.cc:458] T 00000000000000000000000000000000 P d4a332921c23475baf7aeee0e3677960 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 15:16:04.988574 31509 mvcc.cc:205] Tried to move back new op lower bound from 7315930173364727808 to 7315930173304193024. Current Snapshot: MvccSnapshot[applied={T|T < 7315930173364727808 or (T in {7315930173364727808})}]
I20260807 15:16:04.988698 31510 mvcc.cc:205] Tried to move back new op lower bound from 7315930173364727808 to 7315930173304193024. Current Snapshot: MvccSnapshot[applied={T|T < 7315930173364727808}]
I20260807 15:16:04.988983 31343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:16:04.965348 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35158 (local address 127.29.194.61:37549)
0807 15:16:04.965386 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0807 15:16:04.965388 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:16:04.975800 (+ 10412us) server_negotiation.cc:407] Connection header received
0807 15:16:04.975943 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:16:04.975944 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:16:04.975957 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:16:04.975999 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:16:04.977035 (+ 1036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:16:04.977771 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:16:04.978687 (+ 916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:16:04.979627 (+ 940us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:16:04.979673 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:16:04.985631 (+ 5958us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:16:04.985633 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:16:04.985633 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:16:04.985642 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:16:04.987248 (+ 1606us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:16:04.987677 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:16:04.987679 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:16:04.987679 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:16:04.987852 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:16:04.988085 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:16:04.988086 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:16:04.988086 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:16:04.988228 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:16:04.988301 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0807 15:16:04.988716 (+ 415us) server_negotiation.cc:299] Negotiation successful
0807 15:16:04.988888 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260807 15:16:04.997850 31338 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.194.60" port: 39073 } attempt: 3
Aug 07 15:16:05 dist-test-slave-42z9 krb5kdc[31126](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115764, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.194.61@KRBTEST.COM for kudu/127.29.194.60@KRBTEST.COM
I20260807 15:16:05.007324 31342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:16:04.998202 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.60:39073 (local address 127.0.0.1:49040)
0807 15:16:04.998356 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0807 15:16:04.998367 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 15:16:04.998407 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:16:04.998707 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:16:04.998708 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:16:04.998896 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:16:04.998950 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:16:04.998952 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:16:05.000245 (+ 1293us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:16:05.000246 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:16:05.001054 (+ 808us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:16:05.001057 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:16:05.001657 (+ 600us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:16:05.001658 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:16:05.001730 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:16:05.002328 (+ 598us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:16:05.002333 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:16:05.003971 (+ 1638us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:16:05.006106 (+ 2135us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:16:05.006107 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:16:05.006108 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:16:05.006344 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:16:05.006600 (+ 256us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:16:05.006601 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:16:05.006601 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:16:05.006672 (+ 71us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:16:05.006988 (+ 316us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:16:05.006989 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:16:05.007049 (+ 60us) client_negotiation.cc:769] Sending connection context
0807 15:16:05.007107 (+ 58us) client_negotiation.cc:240] Negotiation successful
0807 15:16:05.007213 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117}
I20260807 15:16:05.008175 31447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:16:04.998497 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49040 (local address 127.29.194.60:39073)
0807 15:16:04.998569 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0807 15:16:04.998571 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:16:04.998583 (+ 12us) server_negotiation.cc:407] Connection header received
0807 15:16:04.998607 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:16:04.998608 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:16:04.998619 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:16:04.998681 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:16:04.999013 (+ 332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:16:04.999789 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:16:05.001165 (+ 1376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:16:05.001523 (+ 358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:16:05.001584 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:16:05.004103 (+ 2519us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:16:05.004105 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:16:05.004105 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:16:05.004113 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:16:05.005999 (+ 1886us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:16:05.006452 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:16:05.006453 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:16:05.006453 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:16:05.006525 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:16:05.006742 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:16:05.006743 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:16:05.006743 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:16:05.006886 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:16:05.006951 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0807 15:16:05.007972 (+ 1021us) server_negotiation.cc:299] Negotiation successful
0807 15:16:05.008080 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260807 15:16:05.009946 31338 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e2778131dce245a6a60414bb2b7ddd5a: Bootstrap starting.
I20260807 15:16:05.010725 31338 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e2778131dce245a6a60414bb2b7ddd5a: Neither blocks nor log segments found. Creating new log.
I20260807 15:16:05.011041 31338 log.cc:824] T 00000000000000000000000000000000 P e2778131dce245a6a60414bb2b7ddd5a: Log is configured to *not* fsync() on all Append() calls
I20260807 15:16:05.012245 31338 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e2778131dce245a6a60414bb2b7ddd5a: No bootstrap required, opened a new log
I20260807 15:16:05.014338 31338 raft_consensus.cc:348] T 00000000000000000000000000000000 P e2778131dce245a6a60414bb2b7ddd5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c86b028a2c541be8a44a94a8371b06b" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 40791 } } peers { permanent_uuid: "e2778131dce245a6a60414bb2b7ddd5a" member_type: VOTER last_known_addr { host: "127.29.194.61" port: 37549 } } peers { permanent_uuid: "d4a332921c23475baf7aeee0e3677960" member_type: VOTER last_known_addr { host: "127.29.194.60" port: 39073 } }
I20260807 15:16:05.014485 31338 raft_consensus.cc:729] T 00000000000000000000000000000000 P e2778131dce245a6a60414bb2b7ddd5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2778131dce245a6a60414bb2b7ddd5a, State: Initialized, Role: FOLLOWER
I20260807 15:16:05.014562 31338 consensus_queue.cc:260] T 00000000000000000000000000000000 P e2778131dce245a6a60414bb2b7ddd5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c86b028a2c541be8a44a94a8371b06b" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 40791 } } peers { permanent_uuid: "e2778131dce245a6a60414bb2b7ddd5a" member_type: VOTER last_known_addr { host: "127.29.194.61" port: 37549 } } peers { permanent_uuid: "d4a332921c23475baf7aeee0e3677960" member_type: VOTER last_known_addr { host: "127.29.194.60" port: 39073 } }
I20260807 15:16:05.014912 31338 sys_catalog.cc:564] T 00000000000000000000000000000000 P e2778131dce245a6a60414bb2b7ddd5a [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:16:05.015031 31301 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e2778131dce245a6a60414bb2b7ddd5a [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:16:05.016014 31301 raft_consensus.cc:1261] T 00000000000000000000000000000000 P e2778131dce245a6a60414bb2b7ddd5a [term 1 FOLLOWER]: Refusing update from remote peer 0c86b028a2c541be8a44a94a8371b06b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:16:05.016233 31303 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0c86b028a2c541be8a44a94a8371b06b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "e2778131dce245a6a60414bb2b7ddd5a"
I20260807 15:16:05.017052 31303 raft_consensus.cc:2454] T 00000000000000000000000000000000 P e2778131dce245a6a60414bb2b7ddd5a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0c86b028a2c541be8a44a94a8371b06b in term 1.
I20260807 15:16:05.017593 31473 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 0c86b028a2c541be8a44a94a8371b06b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e2778131dce245a6a60414bb2b7ddd5a" member_type: VOTER last_known_addr { host: "127.29.194.61" port: 37549 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:16:05.019362 31508 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:16:05.020124 31508 catalog_manager.cc:1504] Loading token signing keys...
W20260807 15:16:05.020704 31540 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e2778131dce245a6a60414bb2b7ddd5a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 15:16:05.020768 31540 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 15:16:05.021152 31519 sys_catalog.cc:455] T 00000000000000000000000000000000 P e2778131dce245a6a60414bb2b7ddd5a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0c86b028a2c541be8a44a94a8371b06b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c86b028a2c541be8a44a94a8371b06b" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 40791 } } peers { permanent_uuid: "e2778131dce245a6a60414bb2b7ddd5a" member_type: VOTER last_known_addr { host: "127.29.194.61" port: 37549 } } peers { permanent_uuid: "d4a332921c23475baf7aeee0e3677960" member_type: VOTER last_known_addr { host: "127.29.194.60" port: 39073 } } }
I20260807 15:16:05.021232 31519 sys_catalog.cc:458] T 00000000000000000000000000000000 P e2778131dce245a6a60414bb2b7ddd5a [sys.catalog]: This master's current role is: FOLLOWER
I20260807 15:16:05.021351 31519 sys_catalog.cc:455] T 00000000000000000000000000000000 P e2778131dce245a6a60414bb2b7ddd5a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0c86b028a2c541be8a44a94a8371b06b. Latest consensus state: current_term: 1 leader_uuid: "0c86b028a2c541be8a44a94a8371b06b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c86b028a2c541be8a44a94a8371b06b" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 40791 } } peers { permanent_uuid: "e2778131dce245a6a60414bb2b7ddd5a" member_type: VOTER last_known_addr { host: "127.29.194.61" port: 37549 } } peers { permanent_uuid: "d4a332921c23475baf7aeee0e3677960" member_type: VOTER last_known_addr { host: "127.29.194.60" port: 39073 } } }
I20260807 15:16:05.021405 31519 sys_catalog.cc:458] T 00000000000000000000000000000000 P e2778131dce245a6a60414bb2b7ddd5a [sys.catalog]: This master's current role is: FOLLOWER
I20260807 15:16:05.021517 31519 sys_catalog.cc:455] T 00000000000000000000000000000000 P e2778131dce245a6a60414bb2b7ddd5a [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0c86b028a2c541be8a44a94a8371b06b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c86b028a2c541be8a44a94a8371b06b" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 40791 } } peers { permanent_uuid: "e2778131dce245a6a60414bb2b7ddd5a" member_type: VOTER last_known_addr { host: "127.29.194.61" port: 37549 } } peers { permanent_uuid: "d4a332921c23475baf7aeee0e3677960" member_type: VOTER last_known_addr { host: "127.29.194.60" port: 39073 } } }
I20260807 15:16:05.021569 31519 sys_catalog.cc:458] T 00000000000000000000000000000000 P e2778131dce245a6a60414bb2b7ddd5a [sys.catalog]: This master's current role is: FOLLOWER
I20260807 15:16:05.027805 31534 mvcc.cc:205] Tried to move back new op lower bound from 7315930173364727808 to 7315930173304193024. Current Snapshot: MvccSnapshot[applied={T|T < 7315930173364727808}]
I20260807 15:16:05.027928 31504 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0c86b028a2c541be8a44a94a8371b06b [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0c86b028a2c541be8a44a94a8371b06b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c86b028a2c541be8a44a94a8371b06b" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 40791 } } peers { permanent_uuid: "e2778131dce245a6a60414bb2b7ddd5a" member_type: VOTER last_known_addr { host: "127.29.194.61" port: 37549 } } peers { permanent_uuid: "d4a332921c23475baf7aeee0e3677960" member_type: VOTER last_known_addr { host: "127.29.194.60" port: 39073 } } }
I20260807 15:16:05.028039 31504 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0c86b028a2c541be8a44a94a8371b06b [sys.catalog]: This master's current role is: LEADER
I20260807 15:16:05.029958 31508 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0c86b028a2c541be8a44a94a8371b06b: Generated new TSK 0
I20260807 15:16:05.030224 31508 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260807 15:16:05.059422 31447 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:16:05.053782 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49052 (local address 127.29.194.60:39073)
0807 15:16:05.053832 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0807 15:16:05.053834 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:16:05.058508 (+ 4674us) server_negotiation.cc:407] Connection header received
0807 15:16:05.058624 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:16:05.058625 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:16:05.058638 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:16:05.058669 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:16:05.059329 (+ 660us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49052: BlockingRecv error: recv got EOF from 127.0.0.1:49052 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
W20260807 15:16:05.060281 31238 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:16:05.057769 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47344 (local address 127.29.194.62:40791)
0807 15:16:05.057819 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0807 15:16:05.057821 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:16:05.059711 (+ 1890us) server_negotiation.cc:407] Connection header received
0807 15:16:05.059812 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:16:05.059813 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:16:05.059826 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:16:05.059855 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:16:05.060213 (+ 358us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47344: BlockingRecv error: recv got EOF from 127.0.0.1:47344 (error 108)
Metrics: {"server-negotiator.queue_time_us":25}
W20260807 15:16:05.061082 31343 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:16:05.058172 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35166 (local address 127.29.194.61:37549)
0807 15:16:05.058215 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0807 15:16:05.058216 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 15:16:05.060523 (+ 2307us) server_negotiation.cc:407] Connection header received
0807 15:16:05.060622 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:16:05.060623 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:16:05.060634 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:16:05.060662 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:16:05.061021 (+ 359us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35166: BlockingRecv error: recv got EOF from 127.0.0.1:35166 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260807 15:16:05.063700 31343 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:16:05.062873 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35174 (local address 127.29.194.61:37549)
0807 15:16:05.062910 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0807 15:16:05.062912 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:16:05.063134 (+ 222us) server_negotiation.cc:407] Connection header received
0807 15:16:05.063241 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:16:05.063242 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:16:05.063261 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:16:05.063290 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:16:05.063650 (+ 360us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35174: BlockingRecv error: recv got EOF from 127.0.0.1:35174 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260807 15:16:05.067235 31343 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:16:05.066379 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35176 (local address 127.29.194.61:37549)
0807 15:16:05.066415 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0807 15:16:05.066417 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:16:05.066667 (+ 250us) server_negotiation.cc:407] Connection header received
0807 15:16:05.066767 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:16:05.066768 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:16:05.066779 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:16:05.066808 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:16:05.067184 (+ 376us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35176: BlockingRecv error: recv got EOF from 127.0.0.1:35176 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260807 15:16:05.071949 31343 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:16:05.070532 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35190 (local address 127.29.194.61:37549)
0807 15:16:05.070573 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0807 15:16:05.070575 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:16:05.070821 (+ 246us) server_negotiation.cc:407] Connection header received
0807 15:16:05.070919 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:16:05.070920 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:16:05.070932 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:16:05.071131 (+ 199us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:16:05.071894 (+ 763us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35190: BlockingRecv error: recv got EOF from 127.0.0.1:35190 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260807 15:16:05.078387 31343 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:16:05.077414 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35206 (local address 127.29.194.61:37549)
0807 15:16:05.077479 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0807 15:16:05.077481 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:16:05.077758 (+ 277us) server_negotiation.cc:407] Connection header received
0807 15:16:05.077860 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:16:05.077861 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:16:05.077873 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:16:05.077904 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:16:05.078328 (+ 424us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35206: BlockingRecv error: recv got EOF from 127.0.0.1:35206 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
W20260807 15:16:05.088793 31343 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:16:05.087790 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35220 (local address 127.29.194.61:37549)
0807 15:16:05.087841 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0807 15:16:05.087843 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:16:05.088155 (+ 312us) server_negotiation.cc:407] Connection header received
0807 15:16:05.088267 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:16:05.088268 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:16:05.088281 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:16:05.088313 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:16:05.088729 (+ 416us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35220: BlockingRecv error: recv got EOF from 127.0.0.1:35220 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260807 15:16:05.107299 31343 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:16:05.106288 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35226 (local address 127.29.194.61:37549)
0807 15:16:05.106336 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0807 15:16:05.106338 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:16:05.106647 (+ 309us) server_negotiation.cc:407] Connection header received
0807 15:16:05.106754 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:16:05.106756 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:16:05.106768 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:16:05.106799 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:16:05.107221 (+ 422us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35226: BlockingRecv error: recv got EOF from 127.0.0.1:35226 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
W20260807 15:16:05.141763 31343 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:16:05.140640 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35240 (local address 127.29.194.61:37549)
0807 15:16:05.140690 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0807 15:16:05.140692 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:16:05.140808 (+ 116us) server_negotiation.cc:407] Connection header received
0807 15:16:05.140867 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:16:05.140868 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:16:05.140879 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:16:05.140927 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:16:05.141668 (+ 741us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35240: BlockingRecv error: recv got EOF from 127.0.0.1:35240 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260807 15:16:05.207655 31343 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:16:05.206997 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35252 (local address 127.29.194.61:37549)
0807 15:16:05.207073 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0807 15:16:05.207075 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:16:05.207087 (+ 12us) server_negotiation.cc:407] Connection header received
0807 15:16:05.207139 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:16:05.207140 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:16:05.207153 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:16:05.207201 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:16:05.207588 (+ 387us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35252: BlockingRecv error: recv got EOF from 127.0.0.1:35252 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
W20260807 15:16:05.337895 31343 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:16:05.337035 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35264 (local address 127.29.194.61:37549)
0807 15:16:05.337105 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0807 15:16:05.337107 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:16:05.337122 (+ 15us) server_negotiation.cc:407] Connection header received
0807 15:16:05.337167 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:16:05.337168 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:16:05.337186 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:16:05.337285 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:16:05.337802 (+ 517us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35264: BlockingRecv error: recv got EOF from 127.0.0.1:35264 (error 108)
Metrics: {"server-negotiator.queue_time_us":33}
W20260807 15:16:05.596284 31343 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:16:05.595382 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35272 (local address 127.29.194.61:37549)
0807 15:16:05.595453 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0807 15:16:05.595456 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:16:05.595467 (+ 11us) server_negotiation.cc:407] Connection header received
0807 15:16:05.595549 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:16:05.595550 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:16:05.595562 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:16:05.595607 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:16:05.596125 (+ 518us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35272: BlockingRecv error: recv got EOF from 127.0.0.1:35272 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
I20260807 15:16:05.954882 31469 catalog_manager.cc:1259] Loaded cluster ID: bf222aa5df8845a2a1cdc136cfcb4637
I20260807 15:16:05.954974 31469 catalog_manager.cc:1552] T 00000000000000000000000000000000 P d4a332921c23475baf7aeee0e3677960: loading cluster ID for follower catalog manager: success
I20260807 15:16:05.955812 31469 catalog_manager.cc:1574] T 00000000000000000000000000000000 P d4a332921c23475baf7aeee0e3677960: acquiring CA information for follower catalog manager: success
I20260807 15:16:05.956151 31469 catalog_manager.cc:1602] T 00000000000000000000000000000000 P d4a332921c23475baf7aeee0e3677960: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 15:16:06.021346 31540 catalog_manager.cc:1259] Loaded cluster ID: bf222aa5df8845a2a1cdc136cfcb4637
I20260807 15:16:06.021479 31540 catalog_manager.cc:1552] T 00000000000000000000000000000000 P e2778131dce245a6a60414bb2b7ddd5a: loading cluster ID for follower catalog manager: success
I20260807 15:16:06.022256 31540 catalog_manager.cc:1574] T 00000000000000000000000000000000 P e2778131dce245a6a60414bb2b7ddd5a: acquiring CA information for follower catalog manager: success
I20260807 15:16:06.022516 31540 catalog_manager.cc:1602] T 00000000000000000000000000000000 P e2778131dce245a6a60414bb2b7ddd5a: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 15:16:06.112166 31641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:16:06.109632 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35284 (local address 127.29.194.61:37549)
0807 15:16:06.109750 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0807 15:16:06.109752 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:16:06.109762 (+ 10us) server_negotiation.cc:407] Connection header received
0807 15:16:06.109788 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:16:06.109789 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:16:06.109843 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:16:06.109915 (+ 72us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 15:16:06.110041 (+ 126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:16:06.110533 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:16:06.111338 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:16:06.111545 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:16:06.111601 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:16:06.111788 (+ 187us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 15:16:06.111901 (+ 113us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 15:16:06.111958 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0807 15:16:06.112041 (+ 83us) server_negotiation.cc:299] Negotiation successful
0807 15:16:06.112065 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":97,"thread_start_us":63,"threads_started":1}
I20260807 15:16:06.116735 31652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:16:06.114144 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49054 (local address 127.29.194.60:39073)
0807 15:16:06.114249 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0807 15:16:06.114250 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 15:16:06.114258 (+ 8us) server_negotiation.cc:407] Connection header received
0807 15:16:06.114295 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:16:06.114296 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:16:06.114332 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:16:06.114413 (+ 81us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 15:16:06.114622 (+ 209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:16:06.115119 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:16:06.115918 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:16:06.116115 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:16:06.116178 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:16:06.116397 (+ 219us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 15:16:06.116507 (+ 110us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 15:16:06.116580 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0807 15:16:06.116652 (+ 72us) server_negotiation.cc:299] Negotiation successful
0807 15:16:06.116673 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85,"thread_start_us":52,"threads_started":1}
I20260807 15:16:06.121345 31641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:16:06.118584 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35300 (local address 127.29.194.61:37549)
0807 15:16:06.118639 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0807 15:16:06.118640 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 15:16:06.118869 (+ 229us) server_negotiation.cc:407] Connection header received
0807 15:16:06.118924 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:16:06.118925 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:16:06.118934 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:16:06.118984 (+ 50us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 15:16:06.119094 (+ 110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:16:06.119850 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:16:06.120683 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:16:06.120872 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:16:06.120925 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:16:06.121084 (+ 159us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 15:16:06.121121 (+ 37us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 15:16:06.121173 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0807 15:16:06.121237 (+ 64us) server_negotiation.cc:299] Negotiation successful
0807 15:16:06.121263 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260807 15:16:06.121533 31652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:16:06.118520 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49060 (local address 127.29.194.60:39073)
0807 15:16:06.118557 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0807 15:16:06.118559 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:16:06.118580 (+ 21us) server_negotiation.cc:407] Connection header received
0807 15:16:06.118602 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:16:06.118603 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:16:06.118613 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:16:06.118664 (+ 51us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 15:16:06.118817 (+ 153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:16:06.119442 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:16:06.120315 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:16:06.120608 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:16:06.120673 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:16:06.121209 (+ 536us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 15:16:06.121240 (+ 31us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 15:16:06.121291 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0807 15:16:06.121386 (+ 95us) server_negotiation.cc:299] Negotiation successful
0807 15:16:06.121413 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260807 15:16:06.122460 31661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:16:06.118541 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47350 (local address 127.29.194.62:40791)
0807 15:16:06.118816 (+ 275us) server_negotiation.cc:206] Beginning negotiation
0807 15:16:06.118818 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:16:06.118826 (+ 8us) server_negotiation.cc:407] Connection header received
0807 15:16:06.118873 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:16:06.118874 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:16:06.118925 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:16:06.119002 (+ 77us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 15:16:06.119174 (+ 172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:16:06.119936 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:16:06.120839 (+ 903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:16:06.121070 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:16:06.121133 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:16:06.121916 (+ 783us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 15:16:06.122023 (+ 107us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 15:16:06.122080 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0807 15:16:06.122351 (+ 271us) server_negotiation.cc:299] Negotiation successful
0807 15:16:06.122379 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":251,"thread_start_us":49,"threads_started":1}
I20260807 15:16:06.130956 30472 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7fZSIL/build/release/bin/kudu with pid 31142
I20260807 15:16:06.149220 30472 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7fZSIL/build/release/bin/kudu with pid 31244
I20260807 15:16:06.169149 30472 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7fZSIL/build/release/bin/kudu with pid 31347
2026-08-07T15:16:06Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (3046 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115480851507-30472-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 15:16:06 dist-test-slave-42z9 krb5kdc[31666](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 15:16:06 dist-test-slave-42z9 krb5kdc[31666](info): set up 2 sockets
Aug 07 15:16:06 dist-test-slave-42z9 krb5kdc[31666](info): commencing operation
krb5kdc: starting...
W20260807 15:16:07.321486 30472 mini_kdc.cc:121] Time spent starting KDC: real 1.126s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115480851507-30472-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115480851507-30472-0/krb5kdc/test-user.keytab.
Aug 07 15:16:07 dist-test-slave-42z9 krb5kdc[31666](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115767, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T15:16:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T15:16:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.194.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.194.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.194.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.62.keytab.
Entry for principal kudu/127.29.194.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.62.keytab.
I20260807 15:16:07.458431 30472 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7fZSIL/build/release/bin/kudu
/tmp/dist-test-task7fZSIL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115480851507-30472-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115480851507-30472-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115480851507-30472-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115480851507-30472-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.194.62:44147
--webserver_interface=127.29.194.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.62.keytab
--principal=kudu/127.29.194.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.194.20:34677
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.194.62:44147,127.29.194.61:43897,127.29.194.60:43707
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115480851507-30472-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115480851507-30472-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115480851507-30472-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:16:07.543365 31683 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260807 15:16:07.543605 31683 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:16:07.543632 31683 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:16:07.543659 31683 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:16:07.544816 31683 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:16:07.544864 31683 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:16:07.544883 31683 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:16:07.544900 31683 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:16:07.544915 31683 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:16:07.546497 31683 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.194.20:34677
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115480851507-30472-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115480851507-30472-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.194.62:44147,127.29.194.61:43897,127.29.194.60:43707
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.62.keytab
--principal=kudu/127.29.194.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.194.62:44147
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115480851507-30472-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.194.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31683
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115480851507-30472-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:16:07.546766 31683 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:16:07.546993 31683 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:16:07.552000 31693 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:16:07.552129 31697 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:16:07.552266 31683 server_base.cc:1034] running on GCE node
W20260807 15:16:07.552059 31695 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:16:07.553422 31683 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:16:07.553781 31683 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:16:07.554950 31683 hybrid_clock.cc:648] HybridClock initialized: now 1786115767554934 us; error 43 us; skew 500 ppm
Aug 07 15:16:07 dist-test-slave-42z9 krb5kdc[31666](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115767, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.194.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:16:07.557669 31683 init.cc:377] Logged in from keytab as kudu/127.29.194.62@KRBTEST.COM (short username kudu)
I20260807 15:16:07.558122 31683 webserver.cc:460] Webserver started at http://127.29.194.62:41997/ using document root <none> and password file <none>
I20260807 15:16:07.558342 31683 fs_manager.cc:359] Metadata directory not provided
I20260807 15:16:07.558384 31683 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:16:07.558486 31683 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:16:07.559597 31683 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115480851507-30472-0/minicluster-data/master-0/data/instance:
uuid: "a5b8176cb29e468e81a0313b727a4af5"
format_stamp: "Formatted at 2026-08-07 15:16:07 on dist-test-slave-42z9"
server_key: "b6bfcfa216a2bd8bf6db383a0971772a"
server_key_iv: "8d5a55e1f5772054c49fe7cb530c51bf"
server_key_version: "encryptionkey@0"
I20260807 15:16:07.559911 31683 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115480851507-30472-0/minicluster-data/master-0/wal/instance:
uuid: "a5b8176cb29e468e81a0313b727a4af5"
format_stamp: "Formatted at 2026-08-07 15:16:07 on dist-test-slave-42z9"
server_key: "b6bfcfa216a2bd8bf6db383a0971772a"
server_key_iv: "8d5a55e1f5772054c49fe7cb530c51bf"
server_key_version: "encryptionkey@0"
I20260807 15:16:07.561254 31683 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260807 15:16:07.562184 31706 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:16:07.562530 31683 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260807 15:16:07.562680 31683 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115480851507-30472-0/minicluster-data/master-0/data,/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115480851507-30472-0/minicluster-data/master-0/wal
uuid: "a5b8176cb29e468e81a0313b727a4af5"
format_stamp: "Formatted at 2026-08-07 15:16:07 on dist-test-slave-42z9"
server_key: "b6bfcfa216a2bd8bf6db383a0971772a"
server_key_iv: "8d5a55e1f5772054c49fe7cb530c51bf"
server_key_version: "encryptionkey@0"
I20260807 15:16:07.562800 31683 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115480851507-30472-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115480851507-30472-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115480851507-30472-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:16:07.597194 31683 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:16:07.599588 31683 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:16:07.599757 31683 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:16:07.603945 31683 rpc_server.cc:307] RPC server started. Bound to: 127.29.194.62:44147
I20260807 15:16:07.603995 31787 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.194.62:44147 every 8 connection(s)
I20260807 15:16:07.604378 31683 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115480851507-30472-0/minicluster-data/master-0/data/info.pb
I20260807 15:16:07.604918 30472 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7fZSIL/build/release/bin/kudu as pid 31683
I20260807 15:16:07.605026 30472 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115480851507-30472-0/minicluster-data/master-0/wal/instance
I20260807 15:16:07.605273 30472 external_mini_cluster.cc:1428] Setting key 9c95e5883c8897a1dcf11210235b5d00
I20260807 15:16:07.605530 31788 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:16:07.607159 31788 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.194.62" port: 44147 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 15:16:07 dist-test-slave-42z9 krb5kdc[31666](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115767, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.194.62@KRBTEST.COM for kudu/127.29.194.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.29.194.61@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.194.61@KRBTEST.COM" created.
I20260807 15:16:07.619041 31792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:16:07.607875 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:44147 (local address 127.0.0.1:60474)
0807 15:16:07.608109 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0807 15:16:07.608194 (+ 85us) client_negotiation.cc:174] Beginning negotiation
0807 15:16:07.609072 (+ 878us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:16:07.609535 (+ 463us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:16:07.609539 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:16:07.609906 (+ 367us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:16:07.610126 (+ 220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:16:07.610130 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:16:07.611752 (+ 1622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:16:07.611757 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:16:07.612565 (+ 808us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:16:07.612567 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:16:07.613404 (+ 837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:16:07.613406 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:16:07.613552 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:16:07.614081 (+ 529us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:16:07.614090 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:16:07.615694 (+ 1604us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:16:07.617417 (+ 1723us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:16:07.617424 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:16:07.617426 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:16:07.617682 (+ 256us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:16:07.617886 (+ 204us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:16:07.617887 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:16:07.617887 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:16:07.617970 (+ 83us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:16:07.618437 (+ 467us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:16:07.618445 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:16:07.618514 (+ 69us) client_negotiation.cc:769] Sending connection context
0807 15:16:07.618597 (+ 83us) client_negotiation.cc:240] Negotiation successful
0807 15:16:07.618798 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":109,"threads_started":1}
I20260807 15:16:07.619045 31793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:16:07.608093 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60474 (local address 127.29.194.62:44147)
0807 15:16:07.608231 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0807 15:16:07.608236 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:16:07.608399 (+ 163us) server_negotiation.cc:407] Connection header received
0807 15:16:07.609290 (+ 891us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:16:07.609294 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:16:07.609321 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:16:07.609482 (+ 161us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:16:07.610255 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:16:07.611603 (+ 1348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:16:07.612674 (+ 1071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:16:07.613224 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:16:07.613356 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:16:07.615788 (+ 2432us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:16:07.615796 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:16:07.615796 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:16:07.615803 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:16:07.617325 (+ 1522us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:16:07.617743 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:16:07.617745 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:16:07.617746 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:16:07.617830 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:16:07.618047 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:16:07.618047 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:16:07.618048 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:16:07.618342 (+ 294us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:16:07.618427 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0807 15:16:07.618665 (+ 238us) server_negotiation.cc:299] Negotiation successful
0807 15:16:07.618846 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":93,"server-negotiator.queue_time_us":102,"thread_start_us":50,"threads_started":1}
I20260807 15:16:07.619899 31788 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.194.61" port: 43897 } has no permanent_uuid. Determining permanent_uuid...
I20260807 15:16:07.620263 31792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:16:07.620136 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.61:43897 (local address 127.0.0.1:35472)
0807 15:16:07.620180 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0807 15:16:07.620234 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.194.61:43897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
W20260807 15:16:07.620389 31708 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.29.194.61:43897: connect: Connection refused (error 111)
I20260807 15:16:07.620769 31792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:16:07.620681 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.61:43897 (local address 127.0.0.1:35486)
0807 15:16:07.620722 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0807 15:16:07.620746 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.194.61:43897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260807 15:16:07.620862 31788 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.194.61:43897: Network error: Client connection negotiation failed: client connection to 127.29.194.61:43897: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.194.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.61.keytab.
Entry for principal kudu/127.29.194.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.61.keytab.
I20260807 15:16:07.636987 30472 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7fZSIL/build/release/bin/kudu
/tmp/dist-test-task7fZSIL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115480851507-30472-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115480851507-30472-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115480851507-30472-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115480851507-30472-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.194.61:43897
--webserver_interface=127.29.194.61
--webserver_port=0
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.61.keytab
--principal=kudu/127.29.194.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.194.20:34677
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.194.62:44147,127.29.194.61:43897,127.29.194.60:43707
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115480851507-30472-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115480851507-30472-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115480851507-30472-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 15:16:07.670097 31788 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.194.61" port: 43897 } attempt: 1
I20260807 15:16:07.670578 31792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:16:07.670465 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.61:43897 (local address 127.0.0.1:35494)
0807 15:16:07.670518 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0807 15:16:07.670550 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.194.61:43897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
I20260807 15:16:07.670941 31792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:16:07.670859 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.61:43897 (local address 127.0.0.1:35498)
0807 15:16:07.670902 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0807 15:16:07.670926 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.194.61:43897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260807 15:16:07.671027 31788 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.194.61:43897: Network error: Client connection negotiation failed: client connection to 127.29.194.61:43897: connect: Connection refused (error 111)
W20260807 15:16:07.721539 31796 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260807 15:16:07.721788 31796 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:16:07.721813 31796 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:16:07.721844 31796 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:16:07.723114 31796 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:16:07.723173 31796 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:16:07.723192 31796 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:16:07.723209 31796 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:16:07.723224 31796 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:16:07.724941 31796 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.194.20:34677
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115480851507-30472-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115480851507-30472-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.29.194.62:44147,127.29.194.61:43897,127.29.194.60:43707
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.61.keytab
--principal=kudu/127.29.194.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.194.61:43897
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115480851507-30472-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.29.194.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31796
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115480851507-30472-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:16:07.725219 31796 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:16:07.725497 31796 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:16:07.730757 31808 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:16:07.730757 31806 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:16:07.730811 31796 server_base.cc:1034] running on GCE node
W20260807 15:16:07.730757 31803 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:16:07.731397 31796 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:16:07.731729 31796 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:16:07.732892 31796 hybrid_clock.cc:648] HybridClock initialized: now 1786115767732880 us; error 36 us; skew 500 ppm
Aug 07 15:16:07 dist-test-slave-42z9 krb5kdc[31666](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115767, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.194.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:16:07.736172 31796 init.cc:377] Logged in from keytab as kudu/127.29.194.61@KRBTEST.COM (short username kudu)
I20260807 15:16:07.736675 31796 webserver.cc:460] Webserver started at http://127.29.194.61:46165/ using document root <none> and password file <none>
I20260807 15:16:07.736922 31796 fs_manager.cc:359] Metadata directory not provided
I20260807 15:16:07.736977 31796 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:16:07.737109 31796 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:16:07.738258 31796 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115480851507-30472-0/minicluster-data/master-1/data/instance:
uuid: "f11ce1aac0ed4ddea08db4bec10e5123"
format_stamp: "Formatted at 2026-08-07 15:16:07 on dist-test-slave-42z9"
server_key: "7394888ed1f0f2cbd4de5ebe5aa0609d"
server_key_iv: "4965502d8bc27af526888f5c25228b1e"
server_key_version: "encryptionkey@0"
I20260807 15:16:07.738610 31796 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115480851507-30472-0/minicluster-data/master-1/wal/instance:
uuid: "f11ce1aac0ed4ddea08db4bec10e5123"
format_stamp: "Formatted at 2026-08-07 15:16:07 on dist-test-slave-42z9"
server_key: "7394888ed1f0f2cbd4de5ebe5aa0609d"
server_key_iv: "4965502d8bc27af526888f5c25228b1e"
server_key_version: "encryptionkey@0"
I20260807 15:16:07.739179 31788 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.194.61" port: 43897 } attempt: 2
I20260807 15:16:07.739638 31792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:16:07.739497 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.61:43897 (local address 127.0.0.1:35514)
0807 15:16:07.739565 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0807 15:16:07.739606 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.194.61:43897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260807 15:16:07.740001 31792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:16:07.739904 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.61:43897 (local address 127.0.0.1:35530)
0807 15:16:07.739952 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0807 15:16:07.739979 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.194.61:43897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260807 15:16:07.740084 31788 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.194.61:43897: Network error: Client connection negotiation failed: client connection to 127.29.194.61:43897: connect: Connection refused (error 111)
I20260807 15:16:07.740284 31796 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.002s
I20260807 15:16:07.741053 31818 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:16:07.741197 31796 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260807 15:16:07.741295 31796 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115480851507-30472-0/minicluster-data/master-1/data,/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115480851507-30472-0/minicluster-data/master-1/wal
uuid: "f11ce1aac0ed4ddea08db4bec10e5123"
format_stamp: "Formatted at 2026-08-07 15:16:07 on dist-test-slave-42z9"
server_key: "7394888ed1f0f2cbd4de5ebe5aa0609d"
server_key_iv: "4965502d8bc27af526888f5c25228b1e"
server_key_version: "encryptionkey@0"
I20260807 15:16:07.741362 31796 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115480851507-30472-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115480851507-30472-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115480851507-30472-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:16:07.761852 31796 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:16:07.764314 31796 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:16:07.764484 31796 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:16:07.768625 31796 rpc_server.cc:307] RPC server started. Bound to: 127.29.194.61:43897
I20260807 15:16:07.768673 31894 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.194.61:43897 every 8 connection(s)
I20260807 15:16:07.769017 31796 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115480851507-30472-0/minicluster-data/master-1/data/info.pb
I20260807 15:16:07.770401 31896 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:16:07.772620 31896 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.194.62" port: 44147 } has no permanent_uuid. Determining permanent_uuid...
I20260807 15:16:07.773242 30472 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7fZSIL/build/release/bin/kudu as pid 31796
I20260807 15:16:07.773341 30472 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115480851507-30472-0/minicluster-data/master-1/wal/instance
I20260807 15:16:07.773617 30472 external_mini_cluster.cc:1428] Setting key 59bea2a4fbdad8e1fef47494708a4ab7
Aug 07 15:16:07 dist-test-slave-42z9 krb5kdc[31666](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115767, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.194.61@KRBTEST.COM for kudu/127.29.194.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.29.194.60@KRBTEST.COM; defaulting to no policy
I20260807 15:16:07.784022 31793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:16:07.773819 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60476 (local address 127.29.194.62:44147)
0807 15:16:07.773868 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0807 15:16:07.773870 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:16:07.774233 (+ 363us) server_negotiation.cc:407] Connection header received
0807 15:16:07.774907 (+ 674us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:16:07.774909 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:16:07.774925 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:16:07.774999 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:16:07.775857 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:16:07.776366 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:16:07.777406 (+ 1040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:16:07.777760 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:16:07.777840 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:16:07.780609 (+ 2769us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:16:07.780612 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:16:07.780612 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:16:07.780623 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:16:07.782403 (+ 1780us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:16:07.782885 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:16:07.782886 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:16:07.782886 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:16:07.782943 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:16:07.783240 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:16:07.783243 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:16:07.783244 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:16:07.783445 (+ 201us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:16:07.783528 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0807 15:16:07.783851 (+ 323us) server_negotiation.cc:299] Negotiation successful
0807 15:16:07.783942 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260807 15:16:07.784106 31898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:16:07.773843 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:44147 (local address 127.0.0.1:60476)
0807 15:16:07.774088 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0807 15:16:07.774135 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0807 15:16:07.774788 (+ 653us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:16:07.775059 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:16:07.775062 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:16:07.775387 (+ 325us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:16:07.775721 (+ 334us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:16:07.775728 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:16:07.776467 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:16:07.776468 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:16:07.777301 (+ 833us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:16:07.777303 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:16:07.777859 (+ 556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:16:07.777860 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:16:07.778010 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:16:07.778570 (+ 560us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:16:07.778579 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:16:07.780451 (+ 1872us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:16:07.782518 (+ 2067us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:16:07.782531 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:16:07.782533 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:16:07.782806 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:16:07.782997 (+ 191us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:16:07.782998 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:16:07.782998 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:16:07.783105 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:16:07.783525 (+ 420us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:16:07.783533 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:16:07.783674 (+ 141us) client_negotiation.cc:769] Sending connection context
0807 15:16:07.783769 (+ 95us) client_negotiation.cc:240] Negotiation successful
0807 15:16:07.783911 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":126,"threads_started":1}
I20260807 15:16:07.784693 31896 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.194.61" port: 43897 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.194.60@KRBTEST.COM" created.
Aug 07 15:16:07 dist-test-slave-42z9 krb5kdc[31666](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115767, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.194.61@KRBTEST.COM for kudu/127.29.194.61@KRBTEST.COM
I20260807 15:16:07.792551 31898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:16:07.784971 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.61:43897 (local address 127.0.0.1:35532)
0807 15:16:07.785021 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0807 15:16:07.785031 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 15:16:07.785069 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:16:07.785423 (+ 354us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:16:07.785424 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:16:07.785591 (+ 167us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:16:07.785653 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:16:07.785655 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:16:07.786504 (+ 849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:16:07.786505 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:16:07.787005 (+ 500us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:16:07.787007 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:16:07.787527 (+ 520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:16:07.787528 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:16:07.787601 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:16:07.788001 (+ 400us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:16:07.788006 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:16:07.789578 (+ 1572us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:16:07.791251 (+ 1673us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:16:07.791252 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:16:07.791252 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:16:07.791431 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:16:07.791612 (+ 181us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:16:07.791613 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:16:07.791614 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:16:07.791670 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:16:07.792160 (+ 490us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:16:07.792162 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:16:07.792220 (+ 58us) client_negotiation.cc:769] Sending connection context
0807 15:16:07.792315 (+ 95us) client_negotiation.cc:240] Negotiation successful
0807 15:16:07.792441 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31}
I20260807 15:16:07.792635 31901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:16:07.785076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35532 (local address 127.29.194.61:43897)
0807 15:16:07.785269 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0807 15:16:07.785274 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:16:07.785300 (+ 26us) server_negotiation.cc:407] Connection header received
0807 15:16:07.785322 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:16:07.785324 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:16:07.785338 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:16:07.785416 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:16:07.785725 (+ 309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:16:07.786414 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:16:07.787107 (+ 693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:16:07.787448 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:16:07.787517 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:16:07.789659 (+ 2142us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:16:07.789671 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:16:07.789671 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:16:07.789677 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:16:07.791166 (+ 1489us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:16:07.791505 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:16:07.791507 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:16:07.791508 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:16:07.791558 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:16:07.791719 (+ 161us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:16:07.791720 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:16:07.791721 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:16:07.791869 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:16:07.792185 (+ 316us) server_negotiation.cc:1035] Waiting for connection context
0807 15:16:07.792357 (+ 172us) server_negotiation.cc:299] Negotiation successful
0807 15:16:07.792520 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":67,"server-negotiator.queue_time_us":161,"thread_start_us":115,"threads_started":1}
I20260807 15:16:07.793220 31896 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.194.60" port: 43707 } has no permanent_uuid. Determining permanent_uuid...
I20260807 15:16:07.793606 31898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:16:07.793505 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.60:43707 (local address 127.0.0.1:51840)
0807 15:16:07.793538 (+ 33us) negotiation.cc:107] Waiting for socket to connect
0807 15:16:07.793584 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.194.60:43707: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":15}
W20260807 15:16:07.793717 31824 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.29.194.60:43707: connect: Connection refused (error 111)
I20260807 15:16:07.794163 31898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:16:07.794077 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.60:43707 (local address 127.0.0.1:51842)
0807 15:16:07.794116 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0807 15:16:07.794142 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.194.60:43707: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
W20260807 15:16:07.794265 31896 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.194.60:43707: Network error: Client connection negotiation failed: client connection to 127.29.194.60:43707: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.194.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.60.keytab.
Entry for principal kudu/127.29.194.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.60.keytab.
I20260807 15:16:07.804359 30472 external_mini_cluster.cc:1326] Running /tmp/dist-test-task7fZSIL/build/release/bin/kudu
/tmp/dist-test-task7fZSIL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115480851507-30472-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115480851507-30472-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115480851507-30472-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115480851507-30472-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.194.60:43707
--webserver_interface=127.29.194.60
--webserver_port=0
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.60.keytab
--principal=kudu/127.29.194.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.194.20:34677
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.194.62:44147,127.29.194.61:43897,127.29.194.60:43707
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115480851507-30472-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115480851507-30472-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115480851507-30472-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 15:16:07.831239 31788 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.194.61" port: 43897 } attempt: 3
Aug 07 15:16:07 dist-test-slave-42z9 krb5kdc[31666](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115767, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.194.62@KRBTEST.COM for kudu/127.29.194.61@KRBTEST.COM
I20260807 15:16:07.839978 31901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:16:07.832089 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35540 (local address 127.29.194.61:43897)
0807 15:16:07.832194 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0807 15:16:07.832196 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:16:07.832209 (+ 13us) server_negotiation.cc:407] Connection header received
0807 15:16:07.832276 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:16:07.832278 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:16:07.832292 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:16:07.832342 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:16:07.832825 (+ 483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:16:07.833384 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:16:07.834311 (+ 927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:16:07.834607 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:16:07.834688 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:16:07.837238 (+ 2550us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:16:07.837240 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:16:07.837240 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:16:07.837257 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:16:07.838723 (+ 1466us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:16:07.839194 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:16:07.839195 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:16:07.839195 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:16:07.839257 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:16:07.839461 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:16:07.839461 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:16:07.839461 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:16:07.839598 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:16:07.839652 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0807 15:16:07.839825 (+ 173us) server_negotiation.cc:299] Negotiation successful
0807 15:16:07.839906 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260807 15:16:07.840010 31792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:16:07.831977 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.61:43897 (local address 127.0.0.1:35540)
0807 15:16:07.832083 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0807 15:16:07.832098 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 15:16:07.832194 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:16:07.832356 (+ 162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:16:07.832357 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:16:07.832595 (+ 238us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:16:07.832731 (+ 136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:16:07.832734 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:16:07.833495 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:16:07.833497 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:16:07.834074 (+ 577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:16:07.834076 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:16:07.834704 (+ 628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:16:07.834706 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:16:07.834808 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:16:07.835418 (+ 610us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:16:07.835425 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:16:07.837080 (+ 1655us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:16:07.838824 (+ 1744us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:16:07.838825 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:16:07.838826 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:16:07.839104 (+ 278us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:16:07.839328 (+ 224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:16:07.839329 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:16:07.839329 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:16:07.839406 (+ 77us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:16:07.839656 (+ 250us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:16:07.839657 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:16:07.839721 (+ 64us) client_negotiation.cc:769] Sending connection context
0807 15:16:07.839793 (+ 72us) client_negotiation.cc:240] Negotiation successful
0807 15:16:07.839912 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":72}
I20260807 15:16:07.840574 31788 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.194.60" port: 43707 } has no permanent_uuid. Determining permanent_uuid...
I20260807 15:16:07.840936 31792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:16:07.840830 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.60:43707 (local address 127.0.0.1:51850)
0807 15:16:07.840877 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0807 15:16:07.840913 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.194.60:43707: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
I20260807 15:16:07.841243 31792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:16:07.841158 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.60:43707 (local address 127.0.0.1:51852)
0807 15:16:07.841198 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0807 15:16:07.841225 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.194.60:43707: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260807 15:16:07.841349 31788 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.194.60:43707: Network error: Client connection negotiation failed: client connection to 127.29.194.60:43707: connect: Connection refused (error 111)
I20260807 15:16:07.843497 31896 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.194.60" port: 43707 } attempt: 1
I20260807 15:16:07.843995 31898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:16:07.843802 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.60:43707 (local address 127.0.0.1:51860)
0807 15:16:07.843884 (+ 82us) negotiation.cc:107] Waiting for socket to connect
0807 15:16:07.843948 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.194.60:43707: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260807 15:16:07.844547 31898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:16:07.844359 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.60:43707 (local address 127.0.0.1:51864)
0807 15:16:07.844453 (+ 94us) negotiation.cc:107] Waiting for socket to connect
0807 15:16:07.844512 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.194.60:43707: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
W20260807 15:16:07.844658 31896 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.194.60:43707: Network error: Client connection negotiation failed: client connection to 127.29.194.60:43707: connect: Connection refused (error 111)
I20260807 15:16:07.872524 31788 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.194.60" port: 43707 } attempt: 1
I20260807 15:16:07.873097 31792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:16:07.872943 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.60:43707 (local address 127.0.0.1:51868)
0807 15:16:07.873010 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0807 15:16:07.873060 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.194.60:43707: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260807 15:16:07.873500 31792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:16:07.873388 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.60:43707 (local address 127.0.0.1:51872)
0807 15:16:07.873423 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0807 15:16:07.873473 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.194.60:43707: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
W20260807 15:16:07.873592 31788 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.194.60:43707: Network error: Client connection negotiation failed: client connection to 127.29.194.60:43707: connect: Connection refused (error 111)
W20260807 15:16:07.892405 31907 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260807 15:16:07.892649 31907 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:16:07.892665 31907 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:16:07.892686 31907 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:16:07.893922 31907 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:16:07.893972 31907 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:16:07.893991 31907 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:16:07.894003 31907 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:16:07.894014 31907 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:16:07.895610 31907 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.194.20:34677
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115480851507-30472-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115480851507-30472-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.29.194.62:44147,127.29.194.61:43897,127.29.194.60:43707
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115480851507-30472-0/krb5kdc/kudu_127.29.194.60.keytab
--principal=kudu/127.29.194.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.194.60:43707
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115480851507-30472-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.29.194.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31907
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115480851507-30472-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 14:11:16 UTC on ccycloud.quasar-kwhytv.root.comops.site
build id 4572598
I20260807 15:16:07.895834 31907 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:16:07.896039 31907 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:16:07.901350 31916 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:16:07.901347 31918 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:16:07.901389 31907 server_base.cc:1034] running on GCE node
W20260807 15:16:07.901342 31914 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:16:07.901888 31907 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:16:07.902187 31907 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:16:07.903347 31907 hybrid_clock.cc:648] HybridClock initialized: now 1786115767903334 us; error 30 us; skew 500 ppm
Aug 07 15:16:07 dist-test-slave-42z9 krb5kdc[31666](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115767, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.194.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:16:07.906978 31907 init.cc:377] Logged in from keytab as kudu/127.29.194.60@KRBTEST.COM (short username kudu)
I20260807 15:16:07.907482 31907 webserver.cc:460] Webserver started at http://127.29.194.60:36489/ using document root <none> and password file <none>
I20260807 15:16:07.907706 31907 fs_manager.cc:359] Metadata directory not provided
I20260807 15:16:07.907753 31907 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:16:07.907882 31907 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:16:07.909031 31907 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115480851507-30472-0/minicluster-data/master-2/data/instance:
uuid: "60286997f83c45ff94ecca76ed4cce3c"
format_stamp: "Formatted at 2026-08-07 15:16:07 on dist-test-slave-42z9"
server_key: "c25c8cac855ae2323b4616f50ab921d5"
server_key_iv: "5ff961e5bd503c6fa1bde1af51593e81"
server_key_version: "encryptionkey@0"
I20260807 15:16:07.909405 31907 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115480851507-30472-0/minicluster-data/master-2/wal/instance:
uuid: "60286997f83c45ff94ecca76ed4cce3c"
format_stamp: "Formatted at 2026-08-07 15:16:07 on dist-test-slave-42z9"
server_key: "c25c8cac855ae2323b4616f50ab921d5"
server_key_iv: "5ff961e5bd503c6fa1bde1af51593e81"
server_key_version: "encryptionkey@0"
I20260807 15:16:07.910948 31907 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:16:07.911917 31926 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:16:07.912271 31907 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:16:07.912385 31907 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115480851507-30472-0/minicluster-data/master-2/data,/tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115480851507-30472-0/minicluster-data/master-2/wal
uuid: "60286997f83c45ff94ecca76ed4cce3c"
format_stamp: "Formatted at 2026-08-07 15:16:07 on dist-test-slave-42z9"
server_key: "c25c8cac855ae2323b4616f50ab921d5"
server_key_iv: "5ff961e5bd503c6fa1bde1af51593e81"
server_key_version: "encryptionkey@0"
I20260807 15:16:07.912474 31907 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115480851507-30472-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115480851507-30472-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115480851507-30472-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:16:07.912809 31896 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.194.60" port: 43707 } attempt: 2
I20260807 15:16:07.913291 31898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:16:07.913153 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.60:43707 (local address 127.0.0.1:51880)
0807 15:16:07.913214 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0807 15:16:07.913256 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.194.60:43707: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260807 15:16:07.913642 31898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:16:07.913565 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.60:43707 (local address 127.0.0.1:51884)
0807 15:16:07.913599 (+ 34us) negotiation.cc:107] Waiting for socket to connect
0807 15:16:07.913624 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.194.60:43707: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
W20260807 15:16:07.913728 31896 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.194.60:43707: Network error: Client connection negotiation failed: client connection to 127.29.194.60:43707: connect: Connection refused (error 111)
I20260807 15:16:07.925595 31907 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:16:07.927778 31907 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:16:07.927908 31907 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:16:07.932816 31907 rpc_server.cc:307] RPC server started. Bound to: 127.29.194.60:43707
I20260807 15:16:07.932880 32007 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.194.60:43707 every 8 connection(s)
I20260807 15:16:07.933320 31907 server_base.cc:1166] Dumped server information to /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115480851507-30472-0/minicluster-data/master-2/data/info.pb
I20260807 15:16:07.934661 32008 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:16:07.936388 32008 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.194.62" port: 44147 } has no permanent_uuid. Determining permanent_uuid...
I20260807 15:16:07.940889 30472 external_mini_cluster.cc:1388] Started /tmp/dist-test-task7fZSIL/build/release/bin/kudu as pid 31907
I20260807 15:16:07.941000 30472 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task7fZSIL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115480851507-30472-0/minicluster-data/master-2/wal/instance
I20260807 15:16:07.941205 30472 external_mini_cluster.cc:1428] Setting key e876a686af70c818116c3cdf20930bff
Aug 07 15:16:07 dist-test-slave-42z9 krb5kdc[31666](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115767, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.194.60@KRBTEST.COM for kudu/127.29.194.62@KRBTEST.COM
Aug 07 15:16:07 dist-test-slave-42z9 krb5kdc[31666](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115767, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.194.60@KRBTEST.COM
I20260807 15:16:07.947343 31793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:16:07.937345 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60480 (local address 127.29.194.62:44147)
0807 15:16:07.937475 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0807 15:16:07.937478 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:16:07.937748 (+ 270us) server_negotiation.cc:407] Connection header received
0807 15:16:07.938290 (+ 542us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:16:07.938292 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:16:07.938306 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:16:07.938361 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:16:07.939043 (+ 682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:16:07.939543 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:16:07.940879 (+ 1336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:16:07.941137 (+ 258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:16:07.941202 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:16:07.943963 (+ 2761us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:16:07.943966 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:16:07.943966 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:16:07.943975 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:16:07.945846 (+ 1871us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:16:07.946318 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:16:07.946319 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:16:07.946320 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:16:07.946381 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:16:07.946631 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:16:07.946632 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:16:07.946632 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:16:07.946764 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:16:07.946822 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0807 15:16:07.947138 (+ 316us) server_negotiation.cc:299] Negotiation successful
0807 15:16:07.947231 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260807 15:16:07.947544 32010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:16:07.937281 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.62:44147 (local address 127.0.0.1:60480)
0807 15:16:07.937596 (+ 315us) negotiation.cc:107] Waiting for socket to connect
0807 15:16:07.937643 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0807 15:16:07.938175 (+ 532us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:16:07.938436 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:16:07.938440 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:16:07.938732 (+ 292us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:16:07.938947 (+ 215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:16:07.938952 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:16:07.939636 (+ 684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:16:07.939638 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:16:07.940679 (+ 1041us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:16:07.940692 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:16:07.941262 (+ 570us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:16:07.941263 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:16:07.941481 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:16:07.942206 (+ 725us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:16:07.942217 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:16:07.943882 (+ 1665us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:16:07.945923 (+ 2041us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:16:07.945930 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:16:07.945933 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:16:07.946243 (+ 310us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:16:07.946438 (+ 195us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:16:07.946440 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:16:07.946440 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:16:07.946569 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:16:07.946824 (+ 255us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:16:07.946831 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:16:07.946989 (+ 158us) client_negotiation.cc:769] Sending connection context
0807 15:16:07.947089 (+ 100us) client_negotiation.cc:240] Negotiation successful
0807 15:16:07.947289 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":260,"thread_start_us":169,"threads_started":1}
I20260807 15:16:07.948719 31788 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.194.60" port: 43707 } attempt: 2
I20260807 15:16:07.948657 32008 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.194.61" port: 43897 } has no permanent_uuid. Determining permanent_uuid...
I20260807 15:16:07.951589 32012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:16:07.941746 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51892 (local address 127.29.194.60:43707)
0807 15:16:07.941964 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0807 15:16:07.941969 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:16:07.941988 (+ 19us) server_negotiation.cc:407] Connection header received
0807 15:16:07.942065 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:16:07.942068 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:16:07.942093 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:16:07.942188 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:16:07.942903 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:16:07.943808 (+ 905us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:16:07.944840 (+ 1032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:16:07.945374 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:16:07.945499 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:16:07.947783 (+ 2284us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:16:07.947788 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:16:07.947789 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:16:07.947795 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:16:07.950001 (+ 2206us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:16:07.950642 (+ 641us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:16:07.950646 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:16:07.950647 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:16:07.950731 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:16:07.950969 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:16:07.950969 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:16:07.950970 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:16:07.951174 (+ 204us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:16:07.951342 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0807 15:16:07.951392 (+ 50us) server_negotiation.cc:299] Negotiation successful
0807 15:16:07.951477 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":182,"thread_start_us":108,"threads_started":1}
Aug 07 15:16:07 dist-test-slave-42z9 krb5kdc[31666](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115767, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.194.62@KRBTEST.COM for kudu/127.29.194.60@KRBTEST.COM
Aug 07 15:16:07 dist-test-slave-42z9 krb5kdc[31666](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115767, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.194.60@KRBTEST.COM for kudu/127.29.194.61@KRBTEST.COM
I20260807 15:16:07.957214 32010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:16:07.949084 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.61:43897 (local address 127.0.0.1:35552)
0807 15:16:07.949254 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0807 15:16:07.949264 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 15:16:07.949309 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:16:07.949594 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:16:07.949595 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:16:07.949766 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:16:07.949829 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:16:07.949830 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:16:07.950829 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:16:07.950830 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:16:07.951548 (+ 718us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:16:07.951551 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:16:07.952200 (+ 649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:16:07.952201 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:16:07.952279 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:16:07.952634 (+ 355us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:16:07.952639 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:16:07.954500 (+ 1861us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:16:07.956155 (+ 1655us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:16:07.956156 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:16:07.956157 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:16:07.956369 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:16:07.956570 (+ 201us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:16:07.956571 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:16:07.956571 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:16:07.956638 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:16:07.956897 (+ 259us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:16:07.956898 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:16:07.956946 (+ 48us) client_negotiation.cc:769] Sending connection context
0807 15:16:07.957004 (+ 58us) client_negotiation.cc:240] Negotiation successful
0807 15:16:07.957106 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153}
I20260807 15:16:07.957616 31901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:16:07.949204 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35552 (local address 127.29.194.61:43897)
0807 15:16:07.949250 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0807 15:16:07.949252 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:16:07.949385 (+ 133us) server_negotiation.cc:407] Connection header received
0807 15:16:07.949430 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:16:07.949431 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:16:07.949446 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:16:07.949531 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:16:07.949911 (+ 380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:16:07.950727 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:16:07.951648 (+ 921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:16:07.951963 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:16:07.952031 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:16:07.954582 (+ 2551us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:16:07.954583 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:16:07.954584 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:16:07.954592 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:16:07.956062 (+ 1470us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:16:07.956435 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:16:07.956436 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:16:07.956437 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:16:07.956507 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:16:07.956712 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:16:07.956713 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:16:07.956713 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:16:07.956839 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:16:07.956897 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0807 15:16:07.957407 (+ 510us) server_negotiation.cc:299] Negotiation successful
0807 15:16:07.957525 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260807 15:16:07.957906 31792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:16:07.949005 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.60:43707 (local address 127.0.0.1:51908)
0807 15:16:07.949142 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0807 15:16:07.949154 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 15:16:07.949196 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:16:07.949581 (+ 385us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:16:07.949582 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:16:07.949756 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:16:07.949825 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:16:07.949827 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:16:07.950582 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:16:07.950583 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:16:07.951222 (+ 639us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:16:07.951225 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:16:07.951705 (+ 480us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:16:07.951706 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:16:07.951768 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:16:07.952344 (+ 576us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:16:07.952349 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:16:07.953992 (+ 1643us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:16:07.956718 (+ 2726us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:16:07.956719 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:16:07.956719 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:16:07.956944 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:16:07.957159 (+ 215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:16:07.957159 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:16:07.957160 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:16:07.957228 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:16:07.957588 (+ 360us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:16:07.957590 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:16:07.957640 (+ 50us) client_negotiation.cc:769] Sending connection context
0807 15:16:07.957713 (+ 73us) client_negotiation.cc:240] Negotiation successful
0807 15:16:07.957813 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113}
I20260807 15:16:07.958039 32008 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.194.60" port: 43707 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 15:16:07 dist-test-slave-42z9 krb5kdc[31666](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115767, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.194.62@KRBTEST.COM
I20260807 15:16:07.958467 32013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:16:07.949042 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51908 (local address 127.29.194.60:43707)
0807 15:16:07.949203 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0807 15:16:07.949204 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 15:16:07.949241 (+ 37us) server_negotiation.cc:407] Connection header received
0807 15:16:07.949332 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:16:07.949335 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:16:07.949363 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:16:07.949555 (+ 192us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:16:07.949932 (+ 377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:16:07.950503 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:16:07.951312 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:16:07.951610 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:16:07.951702 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:16:07.954096 (+ 2394us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:16:07.954097 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:16:07.954097 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:16:07.954105 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:16:07.956633 (+ 2528us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:16:07.957028 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:16:07.957029 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:16:07.957029 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:16:07.957097 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:16:07.957309 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:16:07.957310 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:16:07.957311 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:16:07.957488 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:16:07.957598 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0807 15:16:07.958266 (+ 668us) server_negotiation.cc:299] Negotiation successful
0807 15:16:07.958351 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":106,"threads_started":1}
I20260807 15:16:07.960619 31788 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a5b8176cb29e468e81a0313b727a4af5: Bootstrap starting.
I20260807 15:16:07.961489 31793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:16:07.952544 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60482 (local address 127.29.194.62:44147)
0807 15:16:07.952585 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0807 15:16:07.952586 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 15:16:07.952595 (+ 9us) server_negotiation.cc:407] Connection header received
0807 15:16:07.953322 (+ 727us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:16:07.953323 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:16:07.953336 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:16:07.953392 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:16:07.954123 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:16:07.954731 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:16:07.955566 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:16:07.955774 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:16:07.956026 (+ 252us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:16:07.958964 (+ 2938us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:16:07.958966 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:16:07.958966 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:16:07.958973 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:16:07.960263 (+ 1290us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:16:07.960585 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:16:07.960586 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:16:07.960586 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:16:07.960640 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:16:07.960803 (+ 163us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:16:07.960803 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:16:07.960804 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:16:07.960930 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:16:07.961109 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0807 15:16:07.961298 (+ 189us) server_negotiation.cc:299] Negotiation successful
0807 15:16:07.961375 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260807 15:16:07.961555 31788 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a5b8176cb29e468e81a0313b727a4af5: Neither blocks nor log segments found. Creating new log.
I20260807 15:16:07.961881 31788 log.cc:824] T 00000000000000000000000000000000 P a5b8176cb29e468e81a0313b727a4af5: Log is configured to *not* fsync() on all Append() calls
Aug 07 15:16:07 dist-test-slave-42z9 krb5kdc[31666](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115767, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.194.60@KRBTEST.COM for kudu/127.29.194.60@KRBTEST.COM
I20260807 15:16:07.962829 31788 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a5b8176cb29e468e81a0313b727a4af5: No bootstrap required, opened a new log
I20260807 15:16:07.964702 31788 raft_consensus.cc:348] T 00000000000000000000000000000000 P a5b8176cb29e468e81a0313b727a4af5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5b8176cb29e468e81a0313b727a4af5" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 44147 } } peers { permanent_uuid: "f11ce1aac0ed4ddea08db4bec10e5123" member_type: VOTER last_known_addr { host: "127.29.194.61" port: 43897 } } peers { permanent_uuid: "60286997f83c45ff94ecca76ed4cce3c" member_type: VOTER last_known_addr { host: "127.29.194.60" port: 43707 } }
I20260807 15:16:07.964833 31788 raft_consensus.cc:729] T 00000000000000000000000000000000 P a5b8176cb29e468e81a0313b727a4af5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a5b8176cb29e468e81a0313b727a4af5, State: Initialized, Role: FOLLOWER
I20260807 15:16:07.964918 31788 consensus_queue.cc:260] T 00000000000000000000000000000000 P a5b8176cb29e468e81a0313b727a4af5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5b8176cb29e468e81a0313b727a4af5" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 44147 } } peers { permanent_uuid: "f11ce1aac0ed4ddea08db4bec10e5123" member_type: VOTER last_known_addr { host: "127.29.194.61" port: 43897 } } peers { permanent_uuid: "60286997f83c45ff94ecca76ed4cce3c" member_type: VOTER last_known_addr { host: "127.29.194.60" port: 43707 } }
I20260807 15:16:07.965207 32010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:16:07.958288 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.60:43707 (local address 127.0.0.1:51922)
0807 15:16:07.958326 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0807 15:16:07.958337 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 15:16:07.958376 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:16:07.958517 (+ 141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:16:07.958517 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:16:07.958694 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:16:07.958730 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:16:07.958732 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:16:07.959530 (+ 798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:16:07.959530 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:16:07.960221 (+ 691us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:16:07.960224 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:16:07.960658 (+ 434us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:16:07.960659 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:16:07.960745 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:16:07.961100 (+ 355us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:16:07.961106 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:16:07.962483 (+ 1377us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:16:07.964103 (+ 1620us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:16:07.964104 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:16:07.964105 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:16:07.964321 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:16:07.964530 (+ 209us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:16:07.964531 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:16:07.964531 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:16:07.964596 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:16:07.964918 (+ 322us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:16:07.964918 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:16:07.964954 (+ 36us) client_negotiation.cc:769] Sending connection context
0807 15:16:07.964993 (+ 39us) client_negotiation.cc:240] Negotiation successful
0807 15:16:07.965115 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":21}
I20260807 15:16:07.965265 31788 sys_catalog.cc:564] T 00000000000000000000000000000000 P a5b8176cb29e468e81a0313b727a4af5 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:16:07.965219 32015 sys_catalog.cc:455] T 00000000000000000000000000000000 P a5b8176cb29e468e81a0313b727a4af5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5b8176cb29e468e81a0313b727a4af5" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 44147 } } peers { permanent_uuid: "f11ce1aac0ed4ddea08db4bec10e5123" member_type: VOTER last_known_addr { host: "127.29.194.61" port: 43897 } } peers { permanent_uuid: "60286997f83c45ff94ecca76ed4cce3c" member_type: VOTER last_known_addr { host: "127.29.194.60" port: 43707 } } }
I20260807 15:16:07.965430 32015 sys_catalog.cc:458] T 00000000000000000000000000000000 P a5b8176cb29e468e81a0313b727a4af5 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 15:16:07.965549 32012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:16:07.958370 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51922 (local address 127.29.194.60:43707)
0807 15:16:07.958408 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0807 15:16:07.958409 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 15:16:07.958417 (+ 8us) server_negotiation.cc:407] Connection header received
0807 15:16:07.958436 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:16:07.958437 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:16:07.958448 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:16:07.958505 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:16:07.958799 (+ 294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:16:07.959447 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:16:07.960312 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:16:07.960583 (+ 271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:16:07.960884 (+ 301us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:16:07.962557 (+ 1673us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:16:07.962558 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:16:07.962558 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:16:07.962565 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:16:07.964010 (+ 1445us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:16:07.964397 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:16:07.964398 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:16:07.964398 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:16:07.964464 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:16:07.964656 (+ 192us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:16:07.964657 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:16:07.964658 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:16:07.964844 (+ 186us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:16:07.965327 (+ 483us) server_negotiation.cc:1035] Waiting for connection context
0807 15:16:07.965368 (+ 41us) server_negotiation.cc:299] Negotiation successful
0807 15:16:07.965475 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
W20260807 15:16:07.967238 32030 catalog_manager.cc:1558] T 00000000000000000000000000000000 P a5b8176cb29e468e81a0313b727a4af5: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 15:16:07.967350 32030 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 15:16:07.967404 32008 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 60286997f83c45ff94ecca76ed4cce3c: Bootstrap starting.
Aug 07 15:16:07 dist-test-slave-42z9 krb5kdc[31666](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115767, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.194.61@KRBTEST.COM
I20260807 15:16:07.968199 32008 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 60286997f83c45ff94ecca76ed4cce3c: Neither blocks nor log segments found. Creating new log.
I20260807 15:16:07.968537 32008 log.cc:824] T 00000000000000000000000000000000 P 60286997f83c45ff94ecca76ed4cce3c: Log is configured to *not* fsync() on all Append() calls
I20260807 15:16:07.970709 32008 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 60286997f83c45ff94ecca76ed4cce3c: No bootstrap required, opened a new log
I20260807 15:16:07.967970 31683 master_runner.cc:186] Error getting master registration for 127.29.194.61:43897: OK, instance_id { permanent_uuid: "f11ce1aac0ed4ddea08db4bec10e5123" instance_seqno: 1786115767767511 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260807 15:16:07.971426 31683 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260807 15:16:07.972896 31901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:16:07.962663 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35558 (local address 127.29.194.61:43897)
0807 15:16:07.962701 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0807 15:16:07.962703 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:16:07.962711 (+ 8us) server_negotiation.cc:407] Connection header received
0807 15:16:07.962736 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:16:07.962736 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:16:07.962747 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:16:07.962797 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:16:07.963472 (+ 675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:16:07.964199 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:16:07.965108 (+ 909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:16:07.965866 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:16:07.965952 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:16:07.968613 (+ 2661us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:16:07.968615 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:16:07.968615 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:16:07.968624 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:16:07.970759 (+ 2135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:16:07.971229 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:16:07.971230 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:16:07.971231 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:16:07.971293 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:16:07.971922 (+ 629us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:16:07.971924 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:16:07.971924 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:16:07.972091 (+ 167us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:16:07.972269 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0807 15:16:07.972720 (+ 451us) server_negotiation.cc:299] Negotiation successful
0807 15:16:07.972802 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260807 15:16:07.973421 32008 raft_consensus.cc:348] T 00000000000000000000000000000000 P 60286997f83c45ff94ecca76ed4cce3c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5b8176cb29e468e81a0313b727a4af5" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 44147 } } peers { permanent_uuid: "f11ce1aac0ed4ddea08db4bec10e5123" member_type: VOTER last_known_addr { host: "127.29.194.61" port: 43897 } } peers { permanent_uuid: "60286997f83c45ff94ecca76ed4cce3c" member_type: VOTER last_known_addr { host: "127.29.194.60" port: 43707 } }
I20260807 15:16:07.973620 32008 raft_consensus.cc:729] T 00000000000000000000000000000000 P 60286997f83c45ff94ecca76ed4cce3c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60286997f83c45ff94ecca76ed4cce3c, State: Initialized, Role: FOLLOWER
I20260807 15:16:07.973717 32008 consensus_queue.cc:260] T 00000000000000000000000000000000 P 60286997f83c45ff94ecca76ed4cce3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5b8176cb29e468e81a0313b727a4af5" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 44147 } } peers { permanent_uuid: "f11ce1aac0ed4ddea08db4bec10e5123" member_type: VOTER last_known_addr { host: "127.29.194.61" port: 43897 } } peers { permanent_uuid: "60286997f83c45ff94ecca76ed4cce3c" member_type: VOTER last_known_addr { host: "127.29.194.60" port: 43707 } }
I20260807 15:16:07.974074 32008 sys_catalog.cc:564] T 00000000000000000000000000000000 P 60286997f83c45ff94ecca76ed4cce3c [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:16:07.974089 32033 sys_catalog.cc:455] T 00000000000000000000000000000000 P 60286997f83c45ff94ecca76ed4cce3c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5b8176cb29e468e81a0313b727a4af5" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 44147 } } peers { permanent_uuid: "f11ce1aac0ed4ddea08db4bec10e5123" member_type: VOTER last_known_addr { host: "127.29.194.61" port: 43897 } } peers { permanent_uuid: "60286997f83c45ff94ecca76ed4cce3c" member_type: VOTER last_known_addr { host: "127.29.194.60" port: 43707 } } }
I20260807 15:16:07.974280 32033 sys_catalog.cc:458] T 00000000000000000000000000000000 P 60286997f83c45ff94ecca76ed4cce3c [sys.catalog]: This master's current role is: FOLLOWER
I20260807 15:16:07.974849 30472 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 15:16:07.975776 31752 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "a5b8176cb29e468e81a0313b727a4af5"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60482
I20260807 15:16:07.975929 31752 raft_consensus.cc:482] T 00000000000000000000000000000000 P a5b8176cb29e468e81a0313b727a4af5 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260807 15:16:07.975991 31752 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a5b8176cb29e468e81a0313b727a4af5 [term 0 FOLLOWER]: Advancing to term 1
W20260807 15:16:07.976473 32053 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 60286997f83c45ff94ecca76ed4cce3c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 15:16:07.976531 32053 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 15:16:07.977243 31752 raft_consensus.cc:504] T 00000000000000000000000000000000 P a5b8176cb29e468e81a0313b727a4af5 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5b8176cb29e468e81a0313b727a4af5" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 44147 } } peers { permanent_uuid: "f11ce1aac0ed4ddea08db4bec10e5123" member_type: VOTER last_known_addr { host: "127.29.194.61" port: 43897 } } peers { permanent_uuid: "60286997f83c45ff94ecca76ed4cce3c" member_type: VOTER last_known_addr { host: "127.29.194.60" port: 43707 } }
I20260807 15:16:07.977560 31752 leader_election.cc:288] T 00000000000000000000000000000000 P a5b8176cb29e468e81a0313b727a4af5 [CANDIDATE]: Term 1 election: Requested vote from peers f11ce1aac0ed4ddea08db4bec10e5123 (127.29.194.61:43897), 60286997f83c45ff94ecca76ed4cce3c (127.29.194.60:43707)
I20260807 15:16:07.977885 31971 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a5b8176cb29e468e81a0313b727a4af5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "60286997f83c45ff94ecca76ed4cce3c"
I20260807 15:16:07.978003 31971 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 60286997f83c45ff94ecca76ed4cce3c [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:16:07.978870 31971 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 60286997f83c45ff94ecca76ed4cce3c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a5b8176cb29e468e81a0313b727a4af5 in term 1.
I20260807 15:16:07.979364 31711 leader_election.cc:302] T 00000000000000000000000000000000 P a5b8176cb29e468e81a0313b727a4af5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 60286997f83c45ff94ecca76ed4cce3c, a5b8176cb29e468e81a0313b727a4af5; no voters:
I20260807 15:16:07.979497 32015 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a5b8176cb29e468e81a0313b727a4af5 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:16:07.979661 32015 raft_consensus.cc:686] T 00000000000000000000000000000000 P a5b8176cb29e468e81a0313b727a4af5 [term 1 LEADER]: Becoming Leader. State: Replica: a5b8176cb29e468e81a0313b727a4af5, State: Running, Role: LEADER
I20260807 15:16:07.979779 32015 consensus_queue.cc:237] T 00000000000000000000000000000000 P a5b8176cb29e468e81a0313b727a4af5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5b8176cb29e468e81a0313b727a4af5" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 44147 } } peers { permanent_uuid: "f11ce1aac0ed4ddea08db4bec10e5123" member_type: VOTER last_known_addr { host: "127.29.194.61" port: 43897 } } peers { permanent_uuid: "60286997f83c45ff94ecca76ed4cce3c" member_type: VOTER last_known_addr { host: "127.29.194.60" port: 43707 } }
I20260807 15:16:07.980223 32015 sys_catalog.cc:455] T 00000000000000000000000000000000 P a5b8176cb29e468e81a0313b727a4af5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a5b8176cb29e468e81a0313b727a4af5. Latest consensus state: current_term: 1 leader_uuid: "a5b8176cb29e468e81a0313b727a4af5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5b8176cb29e468e81a0313b727a4af5" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 44147 } } peers { permanent_uuid: "f11ce1aac0ed4ddea08db4bec10e5123" member_type: VOTER last_known_addr { host: "127.29.194.61" port: 43897 } } peers { permanent_uuid: "60286997f83c45ff94ecca76ed4cce3c" member_type: VOTER last_known_addr { host: "127.29.194.60" port: 43707 } } }
I20260807 15:16:07.980299 32015 sys_catalog.cc:458] T 00000000000000000000000000000000 P a5b8176cb29e468e81a0313b727a4af5 [sys.catalog]: This master's current role is: LEADER
I20260807 15:16:07.982093 32058 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:16:07.982303 32058 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:16:07.980562 31907 master_runner.cc:186] Error getting master registration for 127.29.194.61:43897: OK, instance_id { permanent_uuid: "f11ce1aac0ed4ddea08db4bec10e5123" instance_seqno: 1786115767767511 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260807 15:16:07.982889 31907 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260807 15:16:07.990684 31971 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 60286997f83c45ff94ecca76ed4cce3c [term 1 FOLLOWER]: Refusing update from remote peer a5b8176cb29e468e81a0313b727a4af5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:16:07.991055 32055 consensus_queue.cc:1035] T 00000000000000000000000000000000 P a5b8176cb29e468e81a0313b727a4af5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "60286997f83c45ff94ecca76ed4cce3c" member_type: VOTER last_known_addr { host: "127.29.194.60" port: 43707 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:16:07.992491 32033 sys_catalog.cc:455] T 00000000000000000000000000000000 P 60286997f83c45ff94ecca76ed4cce3c [sys.catalog]: SysCatalogTable state changed. Reason: New leader a5b8176cb29e468e81a0313b727a4af5. Latest consensus state: current_term: 1 leader_uuid: "a5b8176cb29e468e81a0313b727a4af5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5b8176cb29e468e81a0313b727a4af5" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 44147 } } peers { permanent_uuid: "f11ce1aac0ed4ddea08db4bec10e5123" member_type: VOTER last_known_addr { host: "127.29.194.61" port: 43897 } } peers { permanent_uuid: "60286997f83c45ff94ecca76ed4cce3c" member_type: VOTER last_known_addr { host: "127.29.194.60" port: 43707 } } }
I20260807 15:16:07.992597 32033 sys_catalog.cc:458] T 00000000000000000000000000000000 P 60286997f83c45ff94ecca76ed4cce3c [sys.catalog]: This master's current role is: FOLLOWER
I20260807 15:16:08.001593 31901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:16:07.983232 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35566 (local address 127.29.194.61:43897)
0807 15:16:07.983297 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0807 15:16:07.983299 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:16:07.989699 (+ 6400us) server_negotiation.cc:407] Connection header received
0807 15:16:07.989829 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:16:07.989830 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:16:07.989846 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:16:07.989926 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:16:07.991559 (+ 1633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:16:07.992333 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:16:07.994055 (+ 1722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:16:07.994375 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:16:07.994418 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:16:07.996943 (+ 2525us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:16:07.996944 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:16:07.996944 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:16:07.996952 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:16:07.998561 (+ 1609us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:16:07.999246 (+ 685us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:16:07.999247 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:16:07.999248 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:16:07.999327 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:16:07.999707 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:16:07.999708 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:16:07.999709 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:16:07.999857 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:16:07.999921 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0807 15:16:08.001360 (+ 1439us) server_negotiation.cc:299] Negotiation successful
0807 15:16:08.001490 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260807 15:16:08.003150 31793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:16:07.983501 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60498 (local address 127.29.194.62:44147)
0807 15:16:07.983545 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0807 15:16:07.983547 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:16:07.989900 (+ 6353us) server_negotiation.cc:407] Connection header received
0807 15:16:07.989992 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:16:07.989993 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:16:07.990004 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:16:07.990033 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:16:07.991712 (+ 1679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:16:07.992397 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:16:07.996068 (+ 3671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:16:07.996372 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:16:07.996412 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:16:07.999046 (+ 2634us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:16:07.999049 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:16:07.999049 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:16:07.999057 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:16:08.000789 (+ 1732us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:16:08.001209 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:16:08.001211 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:16:08.001211 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:16:08.001270 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:16:08.001497 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:16:08.001498 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:16:08.001498 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:16:08.001638 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:16:08.001743 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0807 15:16:08.001937 (+ 194us) server_negotiation.cc:299] Negotiation successful
0807 15:16:08.002034 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260807 15:16:08.004868 31896 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.194.60" port: 43707 } attempt: 3
I20260807 15:16:08.005172 32012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:16:07.983654 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51924 (local address 127.29.194.60:43707)
0807 15:16:07.983697 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0807 15:16:07.983699 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:16:07.990848 (+ 7149us) server_negotiation.cc:407] Connection header received
0807 15:16:07.990934 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:16:07.990936 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:16:07.990945 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:16:07.990972 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:16:07.993060 (+ 2088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:16:07.993827 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:16:07.996741 (+ 2914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:16:07.997085 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:16:07.997125 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:16:08.000433 (+ 3308us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:16:08.000435 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:16:08.000435 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:16:08.000443 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:16:08.002521 (+ 2078us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:16:08.003941 (+ 1420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:16:08.003942 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:16:08.003942 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:16:08.004021 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:16:08.004351 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:16:08.004352 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:16:08.004353 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:16:08.004494 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:16:08.004567 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0807 15:16:08.004984 (+ 417us) server_negotiation.cc:299] Negotiation successful
0807 15:16:08.005089 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260807 15:16:08.008710 32015 sys_catalog.cc:455] T 00000000000000000000000000000000 P a5b8176cb29e468e81a0313b727a4af5 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "a5b8176cb29e468e81a0313b727a4af5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5b8176cb29e468e81a0313b727a4af5" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 44147 } } peers { permanent_uuid: "f11ce1aac0ed4ddea08db4bec10e5123" member_type: VOTER last_known_addr { host: "127.29.194.61" port: 43897 } } peers { permanent_uuid: "60286997f83c45ff94ecca76ed4cce3c" member_type: VOTER last_known_addr { host: "127.29.194.60" port: 43707 } } }
I20260807 15:16:08.008817 32015 sys_catalog.cc:458] T 00000000000000000000000000000000 P a5b8176cb29e468e81a0313b727a4af5 [sys.catalog]: This master's current role is: LEADER
I20260807 15:16:08.008834 32033 sys_catalog.cc:455] T 00000000000000000000000000000000 P 60286997f83c45ff94ecca76ed4cce3c [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a5b8176cb29e468e81a0313b727a4af5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5b8176cb29e468e81a0313b727a4af5" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 44147 } } peers { permanent_uuid: "f11ce1aac0ed4ddea08db4bec10e5123" member_type: VOTER last_known_addr { host: "127.29.194.61" port: 43897 } } peers { permanent_uuid: "60286997f83c45ff94ecca76ed4cce3c" member_type: VOTER last_known_addr { host: "127.29.194.60" port: 43707 } } }
I20260807 15:16:08.008895 32065 mvcc.cc:205] Tried to move back new op lower bound from 7315930185686441984 to 7315930185645998080. Current Snapshot: MvccSnapshot[applied={T|T < 7315930185686441984}]
I20260807 15:16:08.008929 32033 sys_catalog.cc:458] T 00000000000000000000000000000000 P 60286997f83c45ff94ecca76ed4cce3c [sys.catalog]: This master's current role is: FOLLOWER
I20260807 15:16:08.009140 32071 mvcc.cc:205] Tried to move back new op lower bound from 7315930185686441984 to 7315930185645998080. Current Snapshot: MvccSnapshot[applied={T|T < 7315930185686441984}]
I20260807 15:16:08.010468 32058 catalog_manager.cc:1347] Generated new cluster ID: 45b3a7c3b67e4ca5a3e46bbb2f517ffe
Aug 07 15:16:08 dist-test-slave-42z9 krb5kdc[31666](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115767, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.194.61@KRBTEST.COM for kudu/127.29.194.60@KRBTEST.COM
I20260807 15:16:08.010524 32058 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:16:08.014081 31898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:16:08.005124 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.194.60:43707 (local address 127.0.0.1:51930)
0807 15:16:08.005211 (+ 87us) negotiation.cc:107] Waiting for socket to connect
0807 15:16:08.005223 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 15:16:08.005268 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:16:08.005431 (+ 163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:16:08.005432 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:16:08.005609 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:16:08.005659 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:16:08.005661 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:16:08.006644 (+ 983us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:16:08.006645 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:16:08.007464 (+ 819us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:16:08.007466 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:16:08.007994 (+ 528us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:16:08.007995 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:16:08.008082 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:16:08.009302 (+ 1220us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:16:08.009308 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:16:08.010950 (+ 1642us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:16:08.012740 (+ 1790us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:16:08.012741 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:16:08.012741 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:16:08.012976 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:16:08.013185 (+ 209us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:16:08.013186 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:16:08.013186 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:16:08.013332 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:16:08.013720 (+ 388us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:16:08.013721 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:16:08.013771 (+ 50us) client_negotiation.cc:769] Sending connection context
0807 15:16:08.013832 (+ 61us) client_negotiation.cc:240] Negotiation successful
0807 15:16:08.013972 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":65}
I20260807 15:16:08.014632 32012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:16:08.005166 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51930 (local address 127.29.194.60:43707)
0807 15:16:08.005239 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0807 15:16:08.005241 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:16:08.005252 (+ 11us) server_negotiation.cc:407] Connection header received
0807 15:16:08.005348 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:16:08.005349 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:16:08.005358 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:16:08.005397 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:16:08.005739 (+ 342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:16:08.006474 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:16:08.007620 (+ 1146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:16:08.007916 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:16:08.007981 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:16:08.011055 (+ 3074us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:16:08.011057 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:16:08.011057 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:16:08.011064 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:16:08.012638 (+ 1574us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:16:08.013060 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:16:08.013061 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:16:08.013061 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:16:08.013121 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:16:08.013404 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:16:08.013405 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:16:08.013406 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:16:08.013627 (+ 221us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:16:08.013691 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0807 15:16:08.014449 (+ 758us) server_negotiation.cc:299] Negotiation successful
0807 15:16:08.014540 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260807 15:16:08.016597 31896 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f11ce1aac0ed4ddea08db4bec10e5123: Bootstrap starting.
I20260807 15:16:08.017344 31896 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f11ce1aac0ed4ddea08db4bec10e5123: Neither blocks nor log segments found. Creating new log.
I20260807 15:16:08.017695 31896 log.cc:824] T 00000000000000000000000000000000 P f11ce1aac0ed4ddea08db4bec10e5123: Log is configured to *not* fsync() on all Append() calls
I20260807 15:16:08.022303 31896 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f11ce1aac0ed4ddea08db4bec10e5123: No bootstrap required, opened a new log
I20260807 15:16:08.026494 31896 raft_consensus.cc:348] T 00000000000000000000000000000000 P f11ce1aac0ed4ddea08db4bec10e5123 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5b8176cb29e468e81a0313b727a4af5" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 44147 } } peers { permanent_uuid: "f11ce1aac0ed4ddea08db4bec10e5123" member_type: VOTER last_known_addr { host: "127.29.194.61" port: 43897 } } peers { permanent_uuid: "60286997f83c45ff94ecca76ed4cce3c" member_type: VOTER last_known_addr { host: "127.29.194.60" port: 43707 } }
I20260807 15:16:08.026925 31896 raft_consensus.cc:729] T 00000000000000000000000000000000 P f11ce1aac0ed4ddea08db4bec10e5123 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f11ce1aac0ed4ddea08db4bec10e5123, State: Initialized, Role: FOLLOWER
I20260807 15:16:08.027158 31896 consensus_queue.cc:260] T 00000000000000000000000000000000 P f11ce1aac0ed4ddea08db4bec10e5123 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5b8176cb29e468e81a0313b727a4af5" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 44147 } } peers { permanent_uuid: "f11ce1aac0ed4ddea08db4bec10e5123" member_type: VOTER last_known_addr { host: "127.29.194.61" port: 43897 } } peers { permanent_uuid: "60286997f83c45ff94ecca76ed4cce3c" member_type: VOTER last_known_addr { host: "127.29.194.60" port: 43707 } }
I20260807 15:16:08.027935 31859 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a5b8176cb29e468e81a0313b727a4af5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "f11ce1aac0ed4ddea08db4bec10e5123"
I20260807 15:16:08.027993 31858 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f11ce1aac0ed4ddea08db4bec10e5123 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:16:08.028577 32058 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:16:08.029199 31858 raft_consensus.cc:1261] T 00000000000000000000000000000000 P f11ce1aac0ed4ddea08db4bec10e5123 [term 1 FOLLOWER]: Refusing update from remote peer a5b8176cb29e468e81a0313b727a4af5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:16:08.029284 32058 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:16:08.029544 32015 consensus_queue.cc:1035] T 00000000000000000000000000000000 P a5b8176cb29e468e81a0313b727a4af5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f11ce1aac0ed4ddea08db4bec10e5123" member_type: VOTER last_known_addr { host: "127.29.194.61" port: 43897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:16:08.030216 31859 raft_consensus.cc:2454] T 00000000000000000000000000000000 P f11ce1aac0ed4ddea08db4bec10e5123 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a5b8176cb29e468e81a0313b727a4af5 in term 1.
I20260807 15:16:08.027877 31896 sys_catalog.cc:564] T 00000000000000000000000000000000 P f11ce1aac0ed4ddea08db4bec10e5123 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:16:08.035521 32058 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a5b8176cb29e468e81a0313b727a4af5: Generated new TSK 0
I20260807 15:16:08.035729 32058 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260807 15:16:08.036695 32099 catalog_manager.cc:1558] T 00000000000000000000000000000000 P f11ce1aac0ed4ddea08db4bec10e5123: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 15:16:08.036760 32099 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 15:16:08.036881 32082 sys_catalog.cc:455] T 00000000000000000000000000000000 P f11ce1aac0ed4ddea08db4bec10e5123 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a5b8176cb29e468e81a0313b727a4af5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5b8176cb29e468e81a0313b727a4af5" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 44147 } } peers { permanent_uuid: "f11ce1aac0ed4ddea08db4bec10e5123" member_type: VOTER last_known_addr { host: "127.29.194.61" port: 43897 } } peers { permanent_uuid: "60286997f83c45ff94ecca76ed4cce3c" member_type: VOTER last_known_addr { host: "127.29.194.60" port: 43707 } } }
I20260807 15:16:08.036967 32082 sys_catalog.cc:458] T 00000000000000000000000000000000 P f11ce1aac0ed4ddea08db4bec10e5123 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 15:16:08.037031 32082 sys_catalog.cc:455] T 00000000000000000000000000000000 P f11ce1aac0ed4ddea08db4bec10e5123 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a5b8176cb29e468e81a0313b727a4af5. Latest consensus state: current_term: 1 leader_uuid: "a5b8176cb29e468e81a0313b727a4af5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5b8176cb29e468e81a0313b727a4af5" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 44147 } } peers { permanent_uuid: "f11ce1aac0ed4ddea08db4bec10e5123" member_type: VOTER last_known_addr { host: "127.29.194.61" port: 43897 } } peers { permanent_uuid: "60286997f83c45ff94ecca76ed4cce3c" member_type: VOTER last_known_addr { host: "127.29.194.60" port: 43707 } } }
I20260807 15:16:08.037081 32082 sys_catalog.cc:458] T 00000000000000000000000000000000 P f11ce1aac0ed4ddea08db4bec10e5123 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 15:16:08.037106 32082 sys_catalog.cc:455] T 00000000000000000000000000000000 P f11ce1aac0ed4ddea08db4bec10e5123 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a5b8176cb29e468e81a0313b727a4af5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5b8176cb29e468e81a0313b727a4af5" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 44147 } } peers { permanent_uuid: "f11ce1aac0ed4ddea08db4bec10e5123" member_type: VOTER last_known_addr { host: "127.29.194.61" port: 43897 } } peers { permanent_uuid: "60286997f83c45ff94ecca76ed4cce3c" member_type: VOTER last_known_addr { host: "127.29.194.60" port: 43707 } } }
I20260807 15:16:08.037142 32082 sys_catalog.cc:458] T 00000000000000000000000000000000 P f11ce1aac0ed4ddea08db4bec10e5123 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 15:16:08.038213 32079 mvcc.cc:205] Tried to move back new op lower bound from 7315930185838731264 to 7315930185645998080. Current Snapshot: MvccSnapshot[applied={T|T < 7315930185686441984}]
I20260807 15:16:08.038591 32055 sys_catalog.cc:455] T 00000000000000000000000000000000 P a5b8176cb29e468e81a0313b727a4af5 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "a5b8176cb29e468e81a0313b727a4af5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5b8176cb29e468e81a0313b727a4af5" member_type: VOTER last_known_addr { host: "127.29.194.62" port: 44147 } } peers { permanent_uuid: "f11ce1aac0ed4ddea08db4bec10e5123" member_type: VOTER last_known_addr { host: "127.29.194.61" port: 43897 } } peers { permanent_uuid: "60286997f83c45ff94ecca76ed4cce3c" member_type: VOTER last_known_addr { host: "127.29.194.60" port: 43707 } } }
I20260807 15:16:08.038712 32055 sys_catalog.cc:458] T 00000000000000000000000000000000 P a5b8176cb29e468e81a0313b727a4af5 [sys.catalog]: This master's current role is: LEADER
W20260807 15:16:08.062580 31793 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:16:08.060694 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60506 (local address 127.29.194.62:44147)
0807 15:16:08.060742 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0807 15:16:08.060745 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:16:08.061839 (+ 1094us) server_negotiation.cc:407] Connection header received
0807 15:16:08.061943 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:16:08.061944 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:16:08.061956 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:16:08.061984 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:16:08.062502 (+ 518us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60506: BlockingRecv error: recv got EOF from 127.0.0.1:60506 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
W20260807 15:16:08.063628 32012 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:16:08.061144 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51932 (local address 127.29.194.60:43707)
0807 15:16:08.061189 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0807 15:16:08.061191 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:16:08.062963 (+ 1772us) server_negotiation.cc:407] Connection header received
0807 15:16:08.063140 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:16:08.063141 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:16:08.063153 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:16:08.063180 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:16:08.063559 (+ 379us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51932: BlockingRecv error: recv got EOF from 127.0.0.1:51932 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
W20260807 15:16:08.064524 31901 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:16:08.061562 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35568 (local address 127.29.194.61:43897)
0807 15:16:08.061602 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0807 15:16:08.061604 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:16:08.063950 (+ 2346us) server_negotiation.cc:407] Connection header received
0807 15:16:08.064044 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:16:08.064046 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:16:08.064057 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:16:08.064097 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:16:08.064422 (+ 325us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35568: BlockingRecv error: recv got EOF from 127.0.0.1:35568 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260807 15:16:08.067404 31901 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:16:08.066204 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35576 (local address 127.29.194.61:43897)
0807 15:16:08.066239 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0807 15:16:08.066241 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:16:08.066472 (+ 231us) server_negotiation.cc:407] Connection header received
0807 15:16:08.066637 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:16:08.066638 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:16:08.066658 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:16:08.066686 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:16:08.067358 (+ 672us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35576: BlockingRecv error: recv got EOF from 127.0.0.1:35576 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260807 15:16:08.070641 31901 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:16:08.069807 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35582 (local address 127.29.194.61:43897)
0807 15:16:08.069842 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0807 15:16:08.069844 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:16:08.070110 (+ 266us) server_negotiation.cc:407] Connection header received
0807 15:16:08.070212 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:16:08.070213 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:16:08.070224 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:16:08.070255 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:16:08.070593 (+ 338us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35582: BlockingRecv error: recv got EOF from 127.0.0.1:35582 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260807 15:16:08.075183 31901 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:16:08.074303 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35590 (local address 127.29.194.61:43897)
0807 15:16:08.074365 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0807 15:16:08.074367 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:16:08.074661 (+ 294us) server_negotiation.cc:407] Connection header received
0807 15:16:08.074753 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:16:08.074754 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:16:08.074764 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:16:08.074792 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:16:08.075132 (+ 340us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35590: BlockingRecv error: recv got EOF from 127.0.0.1:35590 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
W20260807 15:16:08.081691 31901 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:16:08.080702 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35602 (local address 127.29.194.61:43897)
0807 15:16:08.080745 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0807 15:16:08.080747 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:16:08.081016 (+ 269us) server_negotiation.cc:407] Connection header received
0807 15:16:08.081124 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:16:08.081125 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:16:08.081137 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:16:08.081165 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:16:08.081634 (+ 469us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35602: BlockingRecv error: recv got EOF from 127.0.0.1:35602 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260807 15:16:08.092239 31901 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:16:08.091259 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35606 (local address 127.29.194.61:43897)
0807 15:16:08.091302 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0807 15:16:08.091304 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:16:08.091601 (+ 297us) server_negotiation.cc:407] Connection header received
0807 15:16:08.091710 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:16:08.091712 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:16:08.091724 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:16:08.091754 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:16:08.092165 (+ 411us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35606: BlockingRecv error: recv got EOF from 127.0.0.1:35606 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260807 15:16:08.110875 31901 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:16:08.109892 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35612 (local address 127.29.194.61:43897)
0807 15:16:08.109939 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0807 15:16:08.109941 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:16:08.110237 (+ 296us) server_negotiation.cc:407] Connection header received
0807 15:16:08.110343 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:16:08.110344 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:16:08.110356 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:16:08.110386 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:16:08.110813 (+ 427us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35612: BlockingRecv error: recv got EOF from 127.0.0.1:35612 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
W20260807 15:16:08.145479 31901 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:16:08.144713 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35618 (local address 127.29.194.61:43897)
0807 15:16:08.144764 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0807 15:16:08.144766 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:16:08.144916 (+ 150us) server_negotiation.cc:407] Connection header received
0807 15:16:08.144966 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:16:08.144967 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:16:08.144979 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:16:08.145029 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:16:08.145393 (+ 364us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35618: BlockingRecv error: recv got EOF from 127.0.0.1:35618 (error 108)
Metrics: {"server-negotiator.queue_time_us":26}
W20260807 15:16:08.211404 31901 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:16:08.210754 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35622 (local address 127.29.194.61:43897)
0807 15:16:08.210831 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0807 15:16:08.210834 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:16:08.210845 (+ 11us) server_negotiation.cc:407] Connection header received
0807 15:16:08.210917 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:16:08.210918 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:16:08.210929 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:16:08.210973 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:16:08.211342 (+ 369us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35622: BlockingRecv error: recv got EOF from 127.0.0.1:35622 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
W20260807 15:16:08.341282 31901 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:16:08.340648 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35626 (local address 127.29.194.61:43897)
0807 15:16:08.340705 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0807 15:16:08.340707 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:16:08.340775 (+ 68us) server_negotiation.cc:407] Connection header received
0807 15:16:08.340823 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:16:08.340824 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:16:08.340836 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:16:08.340884 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:16:08.341223 (+ 339us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35626: BlockingRecv error: recv got EOF from 127.0.0.1:35626 (error 108)
Metrics: {"server-negotiator.queue_time_us":29}
W20260807 15:16:08.599436 31901 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:16:08.598624 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35634 (local address 127.29.194.61:43897)
0807 15:16:08.598680 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0807 15:16:08.598682 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:16:08.598787 (+ 105us) server_negotiation.cc:407] Connection header received
0807 15:16:08.598823 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:16:08.598824 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:16:08.598837 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:16:08.598884 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:16:08.599355 (+ 471us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35634: BlockingRecv error: recv got EOF from 127.0.0.1:35634 (error 108)
Metrics: {"server-negotiator.queue_time_us":32}
I20260807 15:16:08.976995 32053 catalog_manager.cc:1259] Loaded cluster ID: 45b3a7c3b67e4ca5a3e46bbb2f517ffe
I20260807 15:16:08.977093 32053 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 60286997f83c45ff94ecca76ed4cce3c: loading cluster ID for follower catalog manager: success
I20260807 15:16:08.977944 32053 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 60286997f83c45ff94ecca76ed4cce3c: acquiring CA information for follower catalog manager: success
I20260807 15:16:08.978255 32053 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 60286997f83c45ff94ecca76ed4cce3c: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 15:16:09.037206 32099 catalog_manager.cc:1259] Loaded cluster ID: 45b3a7c3b67e4ca5a3e46bbb2f517ffe
I20260807 15:16:09.037382 32099 catalog_manager.cc:1552] T 00000000000000000000000000000000 P f11ce1aac0ed4ddea08db4bec10e5123: loading cluster ID for follower catalog manager: success
I20260807 15:16:09.038275 32099 catalog_manager.cc:1574] T 00000000000000000000000000000000 P f11ce1aac0ed4ddea08db4bec10e5123: acquiring CA information for follower catalog manager: success
I20260807 15:16:09.038555 32099 catalog_manager.cc:1602] T 00000000000000000000000000000000 P f11ce1aac0ed4ddea08db4bec10e5123: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 15:16:09.115185 32193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:16:09.112646 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35640 (local address 127.29.194.61:43897)
0807 15:16:09.112770 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0807 15:16:09.112772 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:16:09.112782 (+ 10us) server_negotiation.cc:407] Connection header received
0807 15:16:09.112832 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:16:09.112833 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:16:09.112899 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:16:09.112970 (+ 71us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 15:16:09.113118 (+ 148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:16:09.113688 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:16:09.114443 (+ 755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:16:09.114644 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:16:09.114700 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:16:09.114859 (+ 159us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 15:16:09.114974 (+ 115us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 15:16:09.115043 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0807 15:16:09.115066 (+ 23us) server_negotiation.cc:299] Negotiation successful
0807 15:16:09.115091 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":102,"thread_start_us":70,"threads_started":1}
I20260807 15:16:09.119211 32201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:16:09.116982 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51948 (local address 127.29.194.60:43707)
0807 15:16:09.117083 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0807 15:16:09.117084 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 15:16:09.117093 (+ 9us) server_negotiation.cc:407] Connection header received
0807 15:16:09.117136 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:16:09.117136 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:16:09.117175 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:16:09.117248 (+ 73us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 15:16:09.117321 (+ 73us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:16:09.117822 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:16:09.118515 (+ 693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:16:09.118722 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:16:09.118785 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:16:09.118946 (+ 161us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 15:16:09.119046 (+ 100us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 15:16:09.119098 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0807 15:16:09.119126 (+ 28us) server_negotiation.cc:299] Negotiation successful
0807 15:16:09.119146 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82,"thread_start_us":52,"threads_started":1}
I20260807 15:16:09.123734 32193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:16:09.120895 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35650 (local address 127.29.194.61:43897)
0807 15:16:09.120946 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0807 15:16:09.120948 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:16:09.121191 (+ 243us) server_negotiation.cc:407] Connection header received
0807 15:16:09.121230 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:16:09.121231 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:16:09.121243 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:16:09.121315 (+ 72us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 15:16:09.121493 (+ 178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:16:09.122235 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:16:09.123079 (+ 844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:16:09.123272 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:16:09.123327 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:16:09.123472 (+ 145us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 15:16:09.123509 (+ 37us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 15:16:09.123557 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0807 15:16:09.123627 (+ 70us) server_negotiation.cc:299] Negotiation successful
0807 15:16:09.123651 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260807 15:16:09.124357 32210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:16:09.120956 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60508 (local address 127.29.194.62:44147)
0807 15:16:09.121059 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0807 15:16:09.121061 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 15:16:09.121303 (+ 242us) server_negotiation.cc:407] Connection header received
0807 15:16:09.121414 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:16:09.121415 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:16:09.121493 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:16:09.121579 (+ 86us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 15:16:09.121650 (+ 71us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:16:09.122374 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:16:09.123436 (+ 1062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:16:09.123710 (+ 274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:16:09.123768 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:16:09.124007 (+ 239us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 15:16:09.124120 (+ 113us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 15:16:09.124157 (+ 37us) server_negotiation.cc:1035] Waiting for connection context
0807 15:16:09.124246 (+ 89us) server_negotiation.cc:299] Negotiation successful
0807 15:16:09.124272 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82,"thread_start_us":53,"threads_started":1}
I20260807 15:16:09.124990 32201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:16:09.121308 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51958 (local address 127.29.194.60:43707)
0807 15:16:09.121342 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0807 15:16:09.121343 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 15:16:09.122320 (+ 977us) server_negotiation.cc:407] Connection header received
0807 15:16:09.122346 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:16:09.122347 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:16:09.122357 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:16:09.122414 (+ 57us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 15:16:09.122587 (+ 173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:16:09.123321 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:16:09.124208 (+ 887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:16:09.124397 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:16:09.124454 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:16:09.124741 (+ 287us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 15:16:09.124776 (+ 35us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 15:16:09.124828 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0807 15:16:09.124901 (+ 73us) server_negotiation.cc:299] Negotiation successful
0807 15:16:09.124923 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260807 15:16:09.126114 30472 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7fZSIL/build/release/bin/kudu with pid 31683
I20260807 15:16:09.154331 30472 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7fZSIL/build/release/bin/kudu with pid 31796
I20260807 15:16:09.167093 30472 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task7fZSIL/build/release/bin/kudu with pid 31907
2026-08-07T15:16:09Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (2998 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (6045 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (288328 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST